Repository navigation
Get primary and unique keys via replication field interface - #9249
paradox1307 wants to merge 1 commit into
Conversation
|
Loading of only relevant constraints can speed up the linear lookup. |
|
BTW, what is the point to return the position within key? Building Especially this is odd for the (strange) cases when one field is participating in different keys in different positions. |
| BTR_all(m_tdbb, m_relation, *m_idxList, relPages); | ||
| } | ||
|
|
||
| *position = idx_invalid; |
There was a problem hiding this comment.
If nullptr is passed - BOOM!
For what ? |
The current architecture of replication plugin is not designed to execute queries to system tables from the plugin. This can cause the server to hang (at least, we encountered a hang when accessing MON$ tables while executing queries from the plugin). Retrieving PK/Unique from replication field allows to make objects/queries for client that are unaware of the current database structure or do not have access to it. |
Consider the situation when target database has a little different structure than source. You must retrieve PK/UK from target, not source. And as I said - position of the field in the key is not needed. |
|
Debezium-like replication plugins require such information (PK/UK). Therefore, it is useful in any case. It’s not entirely clear to me why the position of fields within keys is necessary. |
Maybe they don't just use is for equality condition/check. |
The only meaningful usage of this information is to cut off non-key fields from old record for update/delete but it is risky as I wrote above. |
It is bad practice to connect (from replication plugin) to target just to find out its structure. The plugin should return control to the server as quickly as possible, and external connection can significantly slow down performance. Also, access to the target is not always possible. Or it might not even exist. In general, the plugin may have no external connections at all. It may only make messages and store them for subsequent processing by some other means. |
Correct.
Some sink-connectors can use field order in message primary keys to create schema in the target database. Adding information about keys to a message makes it more deterministic. |
If the plugin sends data to the target immediately - it have to connect to it anyway. The only available optimization - connection may be established in background beforehand, not in record processing routine(s).
In this case it stores all data and leave processing to "apply" part that do have access to target database.
For this case your implementation is wrong. As I already said, it doesn't work for more than one UK. |
|
Any PR must have clear description of both what it does and why it is necessary. |
I have no idea why the replication plugin on source side could want to execute queries. |
Consider replication of PS: I have no idea why the author wanted to query any monitoring table which are famous with it total lock-up during snapshot build. Deadlock is expected in this case. |
I added motivation for PR to the first comment. Please check. |
The current design kind of assumes that the target side of replication knows the source schema. That limits how far the replication mechanism can go: the replication log ends up "incomplete" - it does not carry the metadata needed to rebuild the table structure on the target side on its own.
If the replication log includes info about which fields make up the PK and UK, the target side can use just the log to:
generate queries and recreate the source table structure as close to the original as possible
identify a record when it changes, even if the target schema doesn’t match the source
create indexes without accessing the source database
With this change, there will be more replication scenarios where logs are self-sufficient carriers of both data and metadata. This will also help integrate independent consumers, not just use replication for database mirroring.
This PR adds the ability to check whether a column is a primary key or a unique key without executing additional queries. Two new methods have been added:
boolean isPrimary(uint* position)and booleanisUnique(uint* position). The first checks whether the column is part of the primary key. The second checks whether it is part of a unique key. If so, both methods returnFB_TRUE(otherwiseFB_FALSE) and also set the column’s position within the key.