Previously, the view contents could become inconsistent with the base tables
in the following scenarios:
1) A concurrent transaction modifies a base table and commits before the
incremental view maintenance starts in the current transaction.
2) A concurrent transaction modifies a base table and commits before the
create_immv or refresh_immv command generates data.
3) Concurrent transactions incrementally update a view with a self-join
or modify multiple base tables simultaneously.
Incremental updates of a view are generally performed sequentially using an
exclusive lock. However, even if we are able to acquire the lock, a concurrent
transaction may have already incrementally updated the view and been committed
before we can acquire it. In REPEATABLE READ or SERIALIZABLE isolation levels,
this could lead to an inconsistent view state, which is the cause of the first
issue.
To fix this, a new field, lastivmupdate, has been added to the pg_ivm_immv
catalog to record the transaction ID of the most recent update to the view.
Before performing view maintenance, the transaction ID is checked. If the
transaction was still in progress at the start of the current transaction,
an error is raised to prevent anomalies.
To fix the second issue, the timing of CreateTrigger() has been moved to
before data generation. This ensures that locks conflicting with table
modifications have been acquired on all base tables. In addition, the latest
snapshot is used in READ COMMITTED level during the data generation to reflect
committed changes from concurrent transactions. Additionally, inconsistencies
that cannot be avoided through locking are prevented by checking the transaction
ID of the last view update, as done for the first issue.
However, concurrent table modifications and create_immv execution still cannot
be detected at the time of view creation. Therefore, create_immv raises a warning
in REPEATABLE READ or SERIALIZABLE isolation levels, suggesting that the command
be used in READ COMMITTED mode or that refresh_immv be executed afterward to
ensure the view remains consistent.
The third issue was caused by the snapshot used for checking tuple visibility in
the table's pre-update state not being the latest one. To fix this, the latest
snapshot is now used in READ COMMITTED mode.
Isolation tests are also added.
Issue #104
Previously, pg_upgrade failed due to the permission denied
because the pg_ivm_immv catalog was in the pg_catalog catalog
(Issue #79). To fix this, all objects created by pg_ivm are
moved to theschema pgivm, which is also created by pg_ivm.
pg_ivm is still not relocatable and this must be installed
to the pgivm schema because the catalog and some internal
functions are referred to unqualified by the schema name
from the pg_ivm module. In future, this might be able to
relocatable during installation, though.
This commit affects compatibility with previous releases.
To allow to access objects like create_immv function as
previous, you need to qualify them with the schema name
or setup search_path properly.
When pg_ivm is dropped, the error "could not open relation with OID ..." occurred
at the hook function that enables DROP TABLE on an IMMV to remove the entry in
pg_ivm_immv. It was because that the primary key was already dropped at the time
pg_ivm_immv's toast is been dropped. Also, DROP TABLE command issued concurrently
with DROP EXTENSION pg_Ivm also could cause the same error because pg_ivm_immv
could be already dropped.
This race condition is fixed by using always RangeVarGetRelidExtended to get OID of
pg_ivm_immv instead of using a cache of get_relname_relid results. This makes sure
that pg_ivm_immv exists when this is scanned.
Previously, DROP EXTENSION pg_ivm failed due to the failure of
opening the index on pg_ivm_immv in PgIvmObjectAccessHook that is
called on dropping pg_ivm_immv, because when pg_ivm_immv is being
dropped, the index on it is already dropped.
This is fixed to return immediately from the hook function if the
dropped table is pg_ivm_immv.
When pg_ivm is installed shared_preload_libraries without executing
CREATE EXTENSION command, the hook function is set while the catalog
table pg_ivm_immv is not created. In this case, the hook function
failed to open the catalog table and an error was raised.
Although this way of installing pg_ivm was not considered, it would
be nice to reduce any possible troubles on users. Therefore, to
prevent this error, check if PgIvmImmvRelationId is invalid before
open it. When this is invalid, pg_ivm_immv relation is not created
yet, so there are not any IMMVs, so we don't have to do anything
in this hook function.
Review and commit message by Yugo Nagata
Build errors/warnings against PostgreSQL 16 are fixed.
Also, adapted to the change of codes, including:
- Get rid of the "new" and "old" entries in a view's rangetable.
(Although, removed codes were dead codes because pg_ivm doesn't
have any rules in pg_rewrite.)
- Rework query relation permission checking
- Require empty Bitmapsets to be represented as NULL
- Fix some compiler warnings
The names of additional columns for aggregate views are derived from column
names specified in the first parameter of create_immv, but when the number
was less than the length of the actual target list, segmentation fault occurred.
Furthermore, when the number of specified columns is more than the target list,
it overrode additional column names and it caused a failure of incremental
maintenance of an aggregate view.
To fix then, check the length of the specified column name list not to access
invalid area, and also prevent from overriding additional column names.
Previously, IMMV including IMMV in its definition can be created by
create_immv(), but it should not be supported by IMMV because we
cannot maintain it recursively for now. This patch prevents it by raising
an error for such view definition on create_immv().
Previously, we used an event trigger and executed DELETE command,
but it caused a privilege error when non-superuser dropped a table
even if it is irrelevant to IMMV.
To fix it, we now use object_access_hook and an entry is deleted
via CatalogTupleDelete which doesn't need the superuser privilege.
Issue #25
In order to re-calculate min/max values for groups where the min
or max value is deleted, we need the view query definition in string
form. However, pg_get_viewdef cannot be used for this purpose because
IMMV's defenition is in pg_ivm_immv but not pg_rewrite. Therefore,
we have to convert query definition in pg_ivm_immv to query
definition string. We can use pg_get_querydef in PG15, but we cannot
in PG14 or earlier, so we use codes in ruleutil.c copied from PG13
or PG14 depending versions.
- Allow to use qualified name
- Confirm if executed by the owner of the IMMV
- Improve the message when specified relation is not an IMMV
- Create a unique index at refresh with no dat if possible
This is actually required, but we want it behave as same
as the pgsql-ivm version for now.
refresh_immv(immv_name, with_data) is a function to refresh IMMV like
REFRESH MATERIALIZED VIEW command. It has two argument.
immv_name is incrementally maintainable materialized view's name, and
with_data is an option that is corresponding to the WITH [NO] DATA option.
When with_data is set false, the IMMV gets unpopulated.
One of differences between IMMVs unpopulated by this function and
normal materialized views unpopulated by REFRESH ... WITH NO DATA
is that such IMMVs can be referenced by SELECT but return no rows,
while unpopulated materialized views are not scanable.
The behaviour may be changed in future to raise an error when unpopulated
an IMMV is scanned.
