With my assistance moving a 5ESS out of the way, I began to focus on the SL-100 again. The next task was to start clearing out the existing configuration of the switch so that I could build it up anew just the way I wanted it.
As you can imagine, the data model for the office data of a DMS-100 family switch is quite complex. There are plenty of dependencies between the tables meaning that the order in which data is entered or deleted is important. As I attempted to go through this process, I ran into a couple of issues that made me re-think how I was going about the process.
This blog post on the topic will be about how NOT to clear out the tables. I'll go through the two big snags I hit and how I got around them, as well as a lot of things I learned along the way.
First, though, I'll give a crash course on how DMS tables work.
Most of the configurable elements of the DMS-100 family switches are configured through entities called tables. If you're familiar with databases, you'll recognize a lot of the information in this section.
Tables are filled with elements called tuples, which you can think of as a row in the table. Each of those tuples has two distinct parts, the key and the value. The key must be unique, and can identify or index the tuple within that table, while the values for each tuple need not be unique.
Both the key and the value may be made up of multiple elements. This can get quite complicated as some tuples are very long and support many different kinds of data for their keys and values.
The simple view, however, is that any tuple is ultimately made up of some number of fields, which can be thought of as columns, where some of the fields are part of the key while the rest of the fields are part of the value. You can think of those fields as the columns of a table.
There is one more complication: a field of a table can be a subtable. A subtable has the same general structure of a table (in that it is filled with tuples consisting of keys and values, each consisting of potentially multiple fields) but each subtable uniquely belongs to a tuple of a higher level table.
A good example of this is the STDPRTCT table, which contains, among others, the STDPRT subtables for each of its tuples. Below is an example of a couple tuples of that table, and the contents of their STDPRT subtables. Note that each tuple in STDPRTCT has its own STDPRT subtable, and the contents of those subtables are not shared between the STDPRTCT tuples.
>lis
EXTPRTNM STDPRT AMAPRT
------------------------
SP27 ( 1) ( 0)
>sub stdprt
>lis all
TOP
FROMDIGS TODIGS
PRETRTE
-------------------------------------------------------
0 5
T DD 1 IBNRTE 30 5 18 NONE
6 6
T DD 1 IBNRTE 51 5 18 NONE
7 7
T DD 1 IBNRTE 46 5 18 NONE
8 8
T DD 1 IBNRTE 42 5 18 NONE
9 9
T DD 1 IBNRTE 30 5 18 NONE
BOTTOM
>ret
SP27 ( 1) ( 0)
>lis 2
EXTPRTNM STDPRT AMAPRT
------------------------
SP27 ( 1) ( 0)
SP16 ( 1) ( 0)
>sub stdprt
>lis all
TOP
FROMDIGS TODIGS
PRETRTE
-------------------------------------------------------
0111 0111
D VACT
0112 0119
T DD 0 IBNRTE 12 10 18 NONE
012 09
T DD 0 IBNRTE 52 10 18 NONE
311 311
T DD 0 IBNRTE 7 3 3 NONE
BOTTOM
>In this example, we can see that the SP27 tuple of STDPRTCT has different data in its STDPRT subtable than the SP16 tuple does.
I started off going through tables one at a time. As I encountered dependencies, the DMS software would alert me that a tuple could not be deleted because some other tuple depended on it. I would then go find the relevant table and start clearing it out first.
This was going fairly smoothly until I hit the HUNTMEM table, which looks like the following:
TOP
HTGRP SEQNO INSERT HTMDATA
-------------------------------------------------------
0 0 N L HOST 03 1 04 14 N
0 1 N L HOST 04 1 00 04 N
0 2 N L HOST 01 1 03 31 N
0 3 N L HOST 09 1 04 03 N
0 4 N L HOST 08 0 10 19 N
0 5 N L HOST 09 0 11 08 N
BOTTOMBy this time, I had developed a couple of scripts for minicom to continually delete tuples until the end of the table was hit. I had been using this script on a few tables so far without any issue.
When I used it on the HUNTMEM table, I got a new message:
WARNING: All additions, deletions, and changes to table HUNTMEM should be entered using SERVORD.Either I did not see this until it was too late, or I did not heed the warning. My script plowed through the table one hunt member at a time until there was just two entries left.
TOP
HTGRP SEQNO INSERT HTMDATA
-------------------------------------------------------
0 0 N L HOST 09 0 11 08 N
0 1 N L HOST 09 0 11 08 N
BOTTOMI don't know if this was exactly what it looked like, but the concept is the same. Somehow, I ended up with two HUNTMEM entries pointing to the same line. The switch would not let me delete one of the tuples because the other tuple was still referencing the line.
The root of the problem ends up being that you're not supposed to delete the first hunt member until all of the others have been deleted. My script, which simply went in order, did not follow this rule. The switch helpfully tried to keep the data model intact by doing some things which ended up putting me into this state.
If I had used SERVORD like the switch suggested, those rules would have been enforced and I would not have ended up in this situation.
After a restart of the switch without saving, I was able to remove the hunt members from within SERVORD and continue on.
At some point, after clearing out a lot of tables, I came to the STDPRTCT table. I had made my way there from another table, like IBNRTE, because there were references into IBNRTE from the STDPRT subtables within STDPRTCT. As such, I had to clear those entries before I could clear IBNRTE.
I don't remember if this was the first table I was dealing with that used subtables. In any case, I decided to delete the tuples of STDPRTCT assuming the subtables would be properly deleted when I did so.
Unfortunately, even after clearing out all of STDPRTCT, IBNRTE still complained that there were references to it from STDPRT! Eventually I deduced the reason.
Shown below is an example of an STDPRT subtable.
TOP
FROMDIGS TODIGS
PRETRTE
-------------------------------------------------------
0 5
T DD 1 IBNRTE 30 5 18 NONE
6 6
T DD 1 IBNRTE 51 5 18 NONE
7 7
T DD 1 IBNRTE 46 5 18 NONE
8 8
T DD 1 IBNRTE 42 5 18 NONE
9 9
T DD 1 IBNRTE 30 5 18 NONE
BOTTOMThese tuples are kind of special because they use what Nortel refers to as a 'digilator' type entry. The FROMDIGS and TODIGS fields are variable length phone number type entries. In the example, the first tuple which ranges FROM 0 TO 5 covers any number which starts with 0, 1, 2, 3, 4, or 5 including numbers longer than a single digit like 000000000000 and 599999999 as examples.
For user viewing and data handling, the tuples are shown as you see above. However, internally the 0 to 5 tuple is really stored as a 0, a 1, a 2, a 3, a 4, and a 5 tuple, where each of those tuples is separate. As a more complex example, if we had a 15 to 3 tuple displayed, internally we would have a 15, 16, 17, 18, 19, 2, and a 3 tuple.
Because of the difference in how the data is presented to the user and stored internally, the semantics of navigating and processing these tuples gets complicated. If I am positioned on the '0 to 5' tuple, really I am positioned on just ONE of the internal tuples, and this may not be apparent from the table editor if I am not aware of this fact.
This especially gets complex when considering deletions.
If I am on the 0 to 5 tuple, and I delete the tuple, I was probably really on the 0 tuple. The 0 tuple gets deleted, while the 1, 2, 3, 4, and 5 tuples remain. To make matters worse, the table editor helpfully positions me after the 5 tuple and on the 6 tuple.
The DMS takes a naive approach to deleting the contents of the STDPRT subtables when I deleted the tuple in the higher order STDPRTCT table that owned them. It simply made one pass deleting each tuple in order just the same as if a user were at the keyboard typing the delete command repeatedly. Because of the strange deletion semantics, in the above example this would successfully delete the 0, 6, 7, 8 and 9 tuples, but leave the 1, 2, 3, 4, and 5 tuples.
Without special handling for this, the DMS then deleted the tuple in STDPRTCT despite its STDPRT subtable not being empty. The tuples that were in the STDPRT subtable still existed but became orphaned with no way to access them from the table editor. However, because the tuples still existed and referenced the IBNRTE table, I still could not clear the IBNRTE table.
This mess really had me scratching my head for a moment until I figured out the odd deletion semantics for these subtable tuples. Once I figured it out, the answer was pretty straightforward:
For each subtable, enter the subtable explicitly and delete all of the tuples. Make sure that once you reach the bottom of the subtable, that it is actually empty. If not, start from the top and run deletion cycles again and again until it is empty. Once it is actually empty, then you can navigate back out to the owning tuple and delete it.
At this point, after I had hit two weird snags like this in as many days, I decided it was time to take a step back and start trying to understand more about the DMS data schema before continuing. I later decided that would take many months or even years and that to do so was the wrong approach simply because of the time it would take.
As it turned out, the first two snags were in fact the only two snags I hit, and as long as I followed the two lessons I had learned, I would be able to clear out all of the switch tables I needed without any other snags. However, I did spend several months trying to understand the DMS data model and learned some interesting things worth noting here.
For one thing, I scripted either the SHOWUSES or the SHOWUSERS command to find out what the DMS thought about the references between all of the tables. I massaged this into a graphviz file and generated this behemoth SVG which is nearly 60000 pixels wide and 3000 high. This was the first sign that this was an extremely complex model that would take a very long time to understand.
I started going through the data manuals like 297-8021-351 which is 12 volumes of about 1000 pages each, slowly looking at all of the relations between the tables. As you can imagine, progress was slow.
I also discovered some very interesting tables on the DMS which seem to contain descriptions of the other tables. This is especially interesting as it seems to imply that all of the information is available to the user to mechanically derive the entire data schema for the DMS.
There are a few relevant tables for that, the first being DDTAB. As far as I can tell, this table defines every single data type in use on the switch in terms of a few primitive elements. More complex types are structures consisting of multiple lesser types.
I also found the KEY_ITEM, KEY_MAP, and LOGICAL_TABLE tables to be very interesting, and I began to put the pieces together a bit.
I began to find more and more of this, looking through the tables. It was becoming clear that there was a lot of metadata visible, all rooted in DDTAB.
I don't have a complete understanding of the whole thing, but I determined based on some reading and poking around, that there seem to be two ways the DMS defines tables.
The first way involves the TABLES table. I think this is the older way. The TABLES table references into the TABTYPES table, which then references the data type for a tuple of that table in DDTAB. That data type will probably be a structure of some sort, and there's probably some kind of convention to determine which part is the key and which part is the value.
The other way involves KEY_ITEM, and LOGICAL_TABLE. KEY_ITEM points into DDTAB for types and also KEY_MAP for mapping procedures (I think the table contains addresses for PROTEL procedures). LOGICAL_TABLE points into DDTAB.
Also possibly involved in some way are the OWNTAB and SYSDATA tables.
Some day, I'd like to dig into all of this more to see if it can be used to fully describe the DMS-100 tables.
