[dns-operations] DNSViz Update: Final Stretch (we hope!)

[Matthew Pounsett]

Hi everyone.  It’s been several weeks since my last update on the current status of the DNSViz historical data, and at least a couple of weeks since I last estimated we’d be up and running.  Here’s a quick summary of what’s been happening, including some of the history for those who have missed it, or have just forgotten since this has been going on so long.

During the transitional period, while we were trying to get the newly delivered database server installed and running, we began  temporarily running the DNSViz database from one of our file servers.  We initially noticed some odd behaviour that was tracked down to a library inconsistency between the two systems, which resulted in some string indexes being treated as corrupt on the second system.  We fixed one broken index, but there were others that we did not spot at the time.  Without the proper constraints in place, some bad data (in the form of duplicate entries for primary key columns) were inserted into the database.

Fast forward to re-importing the database on the old system after it was cleaned up, and several index creation statements and foreign key constraint statements were skipped due to the duplicate records in the database.  This is about where we were the last time I sent an update.

To make matters worse, some of these indexes rely on the existence of other indexes and constraints, which meant that while we were trying to clean things up, our queries resulted in sequential scans of extremely large tables, which took a really long time.  We had to systematically fix all the data, so that the constraints could be properly added, then reinitiate the import of the affected tables (with the fixed data) so that the indexes could then be created.

We believe we’re in the home stretch as we wait for the final import to complete.  However, Casey and I are both, for obvious reasons, very reluctant to try to estimate the time remaining on that process.

I think it’s important to recognize Casey for all the work he’s been putting into helping get this going again.  Since the initial reimport completed and we began cleanup work, as the person who knows the database schema best, Casey has been responsible for virtually all of the forward progress.  While I’ve helped here and there attempting to optimise queries that needed to be run, Casey has been the one doing the actual cleanup work.  I’d very much like to thank him for that.

The signs are good that we should have this back to full functionality soon.  Thanks again to everyone for your patience.

Matt Pounsett
DNS-OARC Systems Engineering



-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Message signed with OpenPGP
URL: <https://lists.dns-oarc.net/pipermail/dns-operations/attachments/20191001/4440a86d/attachment.sig>