T3 Webinar Question and Answer Transcript

A Sign of the Times: Using the ITS Standard (NTCIP 1203) for Dynamic Message Signs (September 26, 2007)

Back to Webinar Files

Tom Stout: Thank you, Ken, for an excellent presentation. Very informative and I really appreciate the work that you've put into it. We do have a number of questions and I will read them and probably pass most of them off to Ken since he is the expert, but I will have to answer a couple of them. The first question is when might the NTCIP 1203 Version 2 be an approved standard?

Ken Vaughn:  Right now it is a recommended standard. That means it should be going out to ballot fairly shortly. I believe they're waiting to wrap up some issues with a reference standard, NTCIP 1201 for global objects, mainly dealing with this daylight savings time issue. But I am guessing-- actually, I believe that working group is meeting today and they're probably listening in on this call, but I'm expecting from what I've seen that that should be moving forward fairly quickly, and the standard should be sent out to vote hopefully by the end of the year, which hopefully means it will be approved by March timeframe. However, it is currently a recommended standard, and the general guidance provided by the NTCIP joint committee is that any changes to a recommended standard will be backwards compatible. So generally speaking, agencies have started deploying equipment based on standards, as soon as they reach the recommended stage, which is what we're at right now.

Q. Tom Stout: Is it sufficient for, under state specifications, simply to make reference to the standard?

Ken Vaughn: No! The standard still has many options available. The reason it has so many options is because, as I mentioned, it may be a tunnel sign for only one line of text. It may not need flashing text or graphics or whatever, so different implementers want to do different things, so there are many options in the standard. What you need to do is go through that protocol requirements list and identify the items you want out of that list. There's a procurement workshop that will discuss that in more detail, and I'm sure if you contact Tom Stout, he'd be happy to set that up.

Tom Stout: That's true.

Q. Tom Stout: Are there now NTCIP testing tools that are officially accepted by FHWA?

Ken Vaughn: Well, I believe the answer to that is, the FHWA does not officially accept any tool. I believe there are a handful of tools available in the industry.

Q. Tom Stout: I would concur with that. I think FHWA's interest is in developing procedures and processes and giving guidance. We certainly do not anticipate any tool development. That's better handled by others, which goes onto the next question that says, "Does FHWA even envision a national testing bed for NTCIP standards testing?" That's a topic that has had a lot of discussion. I don't see FHWA ever developing the role of a national test laboratory, like the laboratory that tests electrical devices and so forth, or the drug, food and so forth. What we do have an interest in doing is to provide the guidance and at least part of the tools processes and procedures that perhaps somewhat could perform that function if they so choose to do it. We don't know how that would be done currently but it's certainly something worth continuing to look at. Ken, do you have anything you want to add to that, since you do a lot of testing?

Ken Vaughn: Not on that particular question, no.

Q. Tom Stout: Does FHWA plan to develop a standard test suite for DMS so that vendors and agencies can verify conformance without the need for consultants?

Ken Vaughn: I think what we've tried to do in this process is to kind of walk that line a little bit of developing a set of test procedures in this XML format that can readily be processed electronically to implement scripts that could be used in private sector tools. So that's a step towards a test suite, I guess. But I would certainly say that anyone doing testing certainly will need a fairly detailed understanding of the standards, because when there are problems, you have to be able to justify why you're pointing out something as a problem. And that requires some detailed expertise. So certainly I think we've improved this, but there's no strict need for consultants, but there is a need for an expert.

Tom Stout: If I could just add to that. There's a lot of-- let me rephrase this. There's a great lack of knowledge within our industry as to how to test things, and that applies to state and local agencies and consultants as well. Our intent is to provide knowledge so that our industry can perform the things that need to be done to implement interoperable devices. And the marketplace will determine where that actually rests.

Q. Tom Stout: Going on to the next question, can a state only include a subset of the standard in the spec?

Ken Vaughn: And the answer there would be yes. That would be done through the protocol requirements list. Well, I guess depending on what the question means. Procurement specifications should include a copy of the PRL, which is a 40 page table out of the standard. It should not include anything else from the standard. In fact, including anything else would basically be a copyright violation, but it can include that table, and then you would highlight the features that you want from that. And as I've mentioned, there's a procurement course that goes into that in detail.

Tom Stout: Yeah, if I can add to that. There are a lot of things that you may not have. You may not have an interest in graphics, for examples, as Virginia had no interest at the time that we did that, so they just did not require those items.

Q. Tom Stout: When will these test procedures finally become available? They are needed as examples, otherwise agencies and their consultants will develop their own.

Tom Stout: The thing that we're waiting for to finalize the documents is the adoption of the Version 2 standard. We want to take what we have to make the modifications that are necessary so that it reflects the current standard, and at that point in time, release them. Ken, do you have anything else to add?

Ken Vaughn: No.

Q. Tom Stout: The next question is kind of along the same lines. What are the FHWA test procedures for Version 2? I've not seen an official set of these test procedures. Where can we get them?
Tom Stout: They're in the future. They will be released some time after the standard is adopted. Next question is, "In your analysis, 75 percent central and 85 percent sign, is this all of the functions in 1203 or a relatively small subset?"

Ken Vaughn: I believe the way those numbers were calculated was, 75 percent of the test procedures were performed on the central, and 85 percent were performed on the sign. That's not directly related to how many of the requirements, because there's a mini to mini mapping there. But I suspect that number is very, very close. And the percentage is essentially 75 percent of the functional requirements defined in the entire standard were addressed. So it is all of the functions in 1203, all of the functional requirements in 1203.

Q. Tom Stout: Please give specific example of test tool problems that occurred during Version 2 early deployment and how they were rectified.

Ken Vaughn: We were actually developing the test tools with the developer there onsite as well. So specific examples, I guess things like CRC errors would arise, and we would identify that there were different interpretations, which is probably a good example. If you had something like a CRC error, the first question is, who is in error? Is it the sign or is it the test tool? You would determine that essentially by some third party analysis. Luckily, some of you may be aware, there's a separate little mini software product provided with exerciser called DMS.EXE where you can type in the information about a message, and it would give you the CRC value. So we would use something like that, identify what the correct value should be, and then compare that to what's being sent and what the sign expected. Then we would just basically say there's an action item on the developer of either the test tool, or of the sign, to go off and fix that. Generally, this process worked very well. There's general agreement on developing a process to identify what the value was supposed to be, identifying who had the wrong value and then that person accepting responsibility for fixing their product.

Q. Tom Stout: The next question I think we've answered: You referenced FHWA test procedures. Are these publicly available?

Tom Stout: Not yet.

Q. Tom Stout: FHWA testing workshop and testing documentation, are these publicly available?

Tom Stout: Not yet.

Q. Tom Stout: Can one enquire that the workshop be given at an agency?

Tom Stout: Yes you can. Drop me an email. That's tom.stout@dot.gov.

Q. Tom Stout: How do the FHWA test tools address integration concerns and needs?

Ken Vaughn: I guess my first comment is – well, the test procedures and the XML are Federal Highway products. Those are FHWA test tools in a sense. The actual software was not a Federal Highway sponsored effort, so just to clarify this, those are not technically authorized Federal Highway tools or anything. As far as how they address integration concerns and needs, I would say primarily is making sure that before you integrate a sign into your system, you want to make sure that that sign is properly speaking NTCIP, not just speaking with your system, because: One, your system may not be speaking proper NTCIP. Two, even within NTCIP, there are options of how you encode things, etc., based on the fact that the NTCIP references other standards, broad industry IT standards that include optional ways when coding things. And using an independent tool is just a much more rigorous process. You test every aspect of that device, as opposed to just the commonly used aspects of an essential system. So the intent is, you do full component testing prior to integration. Once you do integration, then you would develop another set of procedures to ensure that the data being exchanged is still conformant, but those are existing private market tools.

Q. Tom Stout: Next is really a comment. It says, "Yes, DST working group is listening in. Hi, Ken. Good presentation.

Ken Vaughn: Why, thank you.

Q. Tom Stout: When will you have the workshop?

Tom Stout: We have kind of changed our way of doing training workshops. We used to kind of blanket the country if you will. Now what we're looking for is agencies that have a need, they're getting ready to update their procurement documents, or they're getting ready to implement system and they're trying to lay the groundwork for it. If that's the case, then we'd very much like to offer the workshop, because getting the right training at the right time is important. Just contact me and I'll make that happen, assuming I have money.

Q. Tom Stout: Where can an agency obtain an electronic version of the NTCIP 1203 Version 2 recommended standard? They could not find it at http://ntcip.org.

Ken Vaughn: Right. What you have to do, and I just verified this, if you go to the NTCIP website, ntcip.org, and then click on the library link, and then underneath library, a menu pops up. Click on document links. If you click on document links, it provides you a whole listing of all the NTCIP standards, 1203 is one of them. If you click on the 1203 link, not on the shopping cart, but on the 1203 link itself, it takes you to the page of current development status. And at the very bottom of that page, you see a section that talks about what's available, and they talk about bulletins and standards. Underneath standards, there is NTCIP 1203 Version 2, DMS Version 2, and that's dated as of March this year. It is version 02–35A in PDF format that is freely available, and that is the actual recommended standard.

Q.  Ken, it was disturbing to note that the agency still found it difficult to apply Version 2. Your slide noted errors, etc. Would you care to comment?

Ken Vaughn: I think the reference there is in relation to Ashwin's slide, and if the commenter wants to respond back if he's talking about something else, that's fine. But basically it was an issue of filling up the PRL table, in that it's a 40 page table that is-- you just really have to be careful when you're filling it out. The suggestion was made that they turn that into a software sort of application, like Turbo Tax or something. I don't know of any current process to do that, but I guess those are my only comments.

Tom Stout:  Yeah, if I could just add to that briefly, he was also particularly troubled by the difficulty of filling out the values, the parameters for the test cases.

Ken Vaughn: Right. And the challenges there really ended up being-- well, it's an important lesson learned on the project that the process that we used in the test procedures allows users to specify the variables they want to use during a test. So for example, if I'm going to display a message up on the sign, in theory to do a full test, you should pretty well load up that sign display with text so you can see the whole sign is working. But, depending on the sign you've purchased, that might be three characters, or it might be three lines by 20 characters. So the actual message you want to display is going to be dependent on the type of sign you purchased. The way we have kind of resolved this issue, at least in theory at this point-- we've not reduced it to practice on this particular set of procedures-- but the way we have proposed to resolve that is by eliminating a lot of those variables. That the only variables you would have would be directly related to the values you define in your procurement, and that PRL column. The other variables would be removed and would become randomized, and they would be randomized within the parameters set up by either of the standard or your PRL, depending on what sort of variable you were dealing with. So that would address a lot of-- I mean, there's a huge number of variables that he had to fill out. That was problematic, and by just creating the rules to generate those values randomly, would largely solve that problem.

Tom Stout:  Thank you. Are there any plans to update the spec wizard to support this approach with Version 2?

Tom Stout: The short answer is no, not at this time. I think the structure of Spec Wizard, it would take too much work to update, to deal with Version 2. Ken, you're much more familiar with the actual innards of spec wizard than I am, but what would you say?

Ken Vaughn: I think it would be relatively easy to do. I think the key thing is that probably it's not– well, there'd be risks involved if we tried to do it before the standard was at the stage currently or even approved stage. So I can see why it has not been done to date.

Tom Stout: There's another reason too. We couldn't find a lot of evidence, hard evidence, that Spec Wizard was widely used.

Ken Vaughn: Right.

Tom Stout: So given that simple market analysis, we probably would not invest more money into it.

Q.  Tom Stout: Who certifies compliance with the standard?

Ken Vaughn: Well, essentially I guess the answer is, there are private consulting firms who are willing to perform testing according to test procedures. Several of those firms, including my company, will certify that we perform the tests and certify that the results we got are as presented. So if they pass all the tests, they are certified to have passed all the tests. Now that's not quite saying that they are fully conforming with the standard, because as I said, the test procedures are not 100 percent exhaustive, but that takes you a long way there.

Tom Stout: Yeah. There are also some states such as Florida that have certified products lists. They go through a testing process and as Ken pointed out, that doesn't necessarily mean that the item totally conforms to the standard. It's an attempt to bring uniformity within a state.

Q.  Tom Stout: We're probably getting close to the last question: After Version 2, what's next for Version 3?

Ken Vaughn: Oh, golly! One thing that would come to mind offhand is, we have developed test procedures under this project. When those are made available, my guess is, that would be another upgrade that the working group might consider. As far as features or changes or anything, I don't know that the working group is actively thinking about anything right now along those lines. Maybe some corrections. I know there were a few features that had been requested, that they did not produce designs for, such as the trigger mechanism. It was included in a previous draft. It did not make it in the final standard. I don't know if there'll be a push for those features in the future, but right now, I don't know of any major problems.

Tom Stout: I think that pretty much concludes our written comments at this time.

back to top