@@ -1052,8 +1052,8 @@ Intraday Database (IDB)
10521052
10531053The Intraday Database or IDB is a simple process that allows access to
10541054data written down intraday. This assumes that there is an existing WDB
1055- process creating a DB on disk that can be loaded with a simple load command.
1056- As of now default and partbyenum WDB writedown modes are supported.
1055+ (and HDB) process creating a DB on disk that can be loaded with a simple
1056+ load command. As of now default and partbyenum WDB writedown modes are supported.
10571057The responsibility of an IDB is therefore:
10581058
105910591 . Serving queries. Since partbyenum writedown mode is done by enumerated
@@ -1064,6 +1064,59 @@ The responsibility of an IDB is therefore:
106410642 . Can be triggered for a reload. This is usually done by the WDB process
10651065 periodically.
10661066
1067+ ### Considerations
1068+
1069+ The IDB is connected by design and operates in a space between the RDB, WDB and HDB.
1070+ As such, the following are some considerations that must be made when creating an IDB:
1071+
1072+ 1 . The IDB requires both a WDB and HDB to be operational.
1073+
1074+ 2 . On startup, the IDB will automatically attempt to connect itself to a WDB
1075+ to facilitate its initialisations steps. These are:
1076+ - Attempt to connect to the WBD to retrieve required information:
1077+ - WBD location.
1078+ - HDB location.
1079+ - Current partition.
1080+ - Current writedown mode.
1081+ - Register itself with the WDB process.
1082+ - Loads the WDB and the HDB's sym file.
1083+
1084+ 3 . The IDB will have data overlapping with the RDB, so gateways and users should
1085+ be mindful when issuing queries.
1086+
1087+ 4 . The IDB is designed to fail fast so it doesn't affect surrounding components.
1088+ It will self-terminate as soon as an error is encountered, such as when
1089+ attempting to load a corrupted DB.
1090+
1091+ ### Querying an IDB
1092+
1093+ The IDB can be queried just like any other HDB. If writedown mode partbyenum is used it has a useful "maptoint" function which can be used.
1094+ ```
1095+ neg[gwHandle](`.gw.asyncexec;"select from trade where int=maptoint[`GOOG]";`idb);gwHandle[]
1096+ ```
1097+
1098+ ### Scalability
1099+
1100+ The IDB exists on disk and the TorQ gateway will automatically load-balance between
1101+ multiple IDB processes linked to the same IDB on disk. This, alongside the implications
1102+ of shifting data from an RDB to disk while maintaining availability makes the IDB a
1103+ component key to TorQ's scalability.
1104+
1105+ ### Database Reload
1106+
1107+ The IDB can be triggered for an intraday reload, which will automatically detect
1108+ any changes to do the least amount of work possible:
1109+
1110+ 1 . The sym file will only be reloaded if it has changed size on disk.
1111+
1112+ 2 . In partbyenum writedown mode, the IDB will only be reloaded if the number
1113+ of partitions has changed.
1114+
1115+ 3 . A reload can be forced in cases where the above conditions are not guaranteed
1116+ to be met.
1117+
1118+ The WDB process triggers the intraday reload on all IDBs registered to it.
1119+
10671120![ IDB diagram] ( graphics/idb.png )
10681121
10691122<a name =" tick " ></a >
0 commit comments