Module 56 - T351

T351: Center-to-Center (C2C) Reference Implementation (RI): Applying the C2C Reference Implementation

HTML of the Course Transcript

(Note: This document has been converted from the transcript to 508-compliant HTML. The formatting has been adjusted for 508 compliance, but all the original text content is included.)

 

Ken Leonard: ITS Standards can make your life easier. Your procurements will go more smoothly and you'll encourage competition, but only if you know how to write them into your specifications and test them. This module is one in a series that covers practical applications for acquiring and testing standards-based ITS systems.

I am Ken Leonard, director of the ITS Joint Program Office for USDOT, and I want to welcome you to our newly redesigned ITS standards training program of which this module is a part. We are pleased to be working with our partner, the Institute of Transportation Engineers, to deliver this new approach to training that combines web-based modules with instructor interaction to bring the latest in ITS learning to busy professionals like yourself.

This combined approach allows interested professionals to schedule training at your convenience, without the need to travel. After you complete this training, we hope that you will tell colleagues and customers about the latest ITS standards and encourage them to take advantage of the archived version of the webinars.

ITS Standards training is one of the first offerings of our updated Professional Capacity Training Program. Through the PCB program we prepare professionals to adopt proven and emerging ITS technologies that will make surface transportation safer, smarter, and greener, which improves livability for us all. You can find information on additional modules and training programs on our website www.pcb.its.dot.gov.

Please help us make even more improvements to our training modules through the evaluation process. We look forward to hearing your comments. Thank you again for participating and we hope you find this module helpful.

Ken Vaughn: Hi, this is Ken Vaughn and we're here to talk about course T351 today. It is the Center-to-Center Reference Implementation Applying the C2C Reference Implementation.

Ken Vaughn: I am your presenter today, Kenneth Vaughn. I'm the president of Trevilon LLC. I've developed test software to verify device conformance in NTCIP and also I've done testing services for the industry over a decade, so I'm well experienced with these testing issues and I'll be discussing these topics with you today.

Ken Vaughn: The course has four learning objectives. The first learning objective is learning how to install and configure the C2C RI on a PC. The second one will discuss how to operate the C2C RI software itself. Then we'll go on and retrieve the C2C RI results from a sample test, and then finally prepare a report based on the C2C RI results. Now, from these learning objectives, this is kind of the advanced course. It's actually how to use the software, so we recommend this for users who really want to get into the details and have quite a bit of experience in understanding how center-to-center communications works.

Ken Vaughn: With that, we'll go ahead and discuss a little bit about Learning Objective 1, in installing, configuring the C2C RI on a personal computer.

Ken Vaughn: The first thing you have to do is obtain the software. The good news is that you can obtain the software for free from the government website at https://www.standards.its.dot.gov/DeploymentResources/Tools. Once again, that software is free and it's updated periodically with the latest version, and that download will actually include the User Manual for it as well. So that User Manual in some areas goes a little bit deeper than this course does, but this course provided several different videos that will assist you in understanding how the system really works. Also, if you need to, while you're using the software, you can obtain technical support via email using the email address of c2crisupport@transcore.com.

Ken Vaughn: Now in order to run the software that you download, you'll need to install it on a machine. That machine will have to have certain system requirements that we'll discuss. There's also some key interoperability requirements, and then we'll discuss the skill set requirements for the operator him or herself.

Ken Vaughn: So the first thing we need to look at are the minimum system requirements. We do recommend either Windows 7 or 8. We have actually tested this on-- well, we've not tested it on Windows 10 officially. I've ran it myself on Windows 10 and it seemed to work reasonably well, but it's only been officially tested on Windows 7 or 8, and this is the 64-bit Professional version of these operating systems. So we also recommend a 2 gigahertz processor, 4 gigabytes of RAM, 1 gigabyte of storage, 1 gigabit-per-second Ethernet connection, and finally, the Java SE Runtime Environment JRE V7.17.

Ken Vaughn: The purpose of the software is to ensure that your systems are interoperable. Now, what precisely does interoperability mean? It's the main purpose of ITS standards, but what really do we mean by that? Well, IEEE has a good definition. It is: "The ability of two or more systems or components to exchange information and use the information that has been exchanged." As a simple example, consider two systems exchange comma-separated values, CSV type files. If the receiving system is a simple cloud storage system and does not know how to read the CSV file, the two systems could not be considered interoperable because the receiving system is unable to use the information. But interoperability is realized when another system is able to read and use the CSV file, connects to the cloud service, and downloads the file. The cloud storage system still offers useful services that's not interoperable, but you don't gain interoperability until you have another system that actually can use that information. So that's what we mean by interoperability, is not only exchanging the information but you actually have application logic than can use that information.

Ken Vaughn: That's what we try to do somewhat with the C2C RI and certainly with the standards within ITS. So, what are the interoperability requirements? Well, C2C interoperability testing requires several different things. In order to realize the desired level of interoperability, we must define our needs and specify the desired system with requirements. In other words, in our previous example, does the receiving system just parse the CSV file and add it to its database, or is the receiving system supposedly perform an operation based on the information? Needs and requirements define precisely what result is expected from interoperability and the requirements claim conformance to various features. Standardized needs and requirements are defined in the TMDD. Modules A321a and A321b provide guidance on how to specify optional features from these standards. We also need a precise definition of the details used to satisfy those requirements. That's provided in the TMDD and NTCIP 2306 standards. They provide a precise design for each requirement along with traceability back to the standardized requirements and user needs. But simply specifying how it works isn't really far enough. We need to go further and identify how we verify and validate the system, and then that verification and validation needs to be performed. So Module T321 defines that process, and the C2C RI extends this through Modules T251 and T351 in identifying how the software tools assists in that verification stage. Now, it should be noted that the C2C RI focuses on verification. Validation is something you do on the end system. One of the key things that we need to remember is the C2C RI and testing in general, we provide essentially spot checks. The number of possibilities are almost endless with this testing, so we can't perform exhaustive testing. It'd simply require too much time. The C2C RI is currently designed to check the most obvious issue and the tool will likely be enhanced over time based on user feedback. Finally, passing C2C RI does not guarantee full interoperability because of the fact that we're only testing what we can in a reasonable amount of time. It does require user interaction at points and it is not exhaustive. Finally, standards themselves may have ambiguities and so an error within the testing of the C2C RI might reflect some other issue. So every issue identified by the tool needs to be thoroughly investigated to identify what portion of the system is responsible for that problem. Is it the standard? Is it the C2C RI tool itself, or is it the system under test, a user error, etc.?

Ken Vaughn: Now with that, and in order to do that sort of analysis, the operator of the test software needs to have quite a bit of experience and knowledge. We'll go over these in detail in Module T251, but we summarize them here, and that is there needs to have a background in encoding languages. The user needs to be familiar with things like XML, because the user will need to actually create XML documents and interpret the message contents in many of these tests. The user also needs to be familiar with protocols to understand what the tester is doing, and in order to resolve problems in the test setup. This also helps them in debugging the process when looking at logs and understanding what went wrong when. Next, the user needs to be familiar with Windows networking and be able to properly configure the test to ensure that the message is being sent across the network. Then of course he has to be familiar with the ITS standards and know which requirements to check and to verify that failures detected by the tool are in fact issues of nonconformance. Then there's really-- this is a fairly advanced set of tests that are being performed, so the user really should have some testing experience and know how to perform repeatable tests in a methodical manner according to industry standards and to document that process along the way. The scripting language is perhaps somewhat optional. This is only necessary if the user plans to test things that are not part of the standardized features that have already been implemented within the C2C RI. But if he wants to extend it to project-specific extensions, he would need to expand the tool, and to do that he would have to write his own scripts according to the process. Now we don't discuss that a lot in this course. There is guidance on how to write those scripting scripts within the user documentation. The final step is understanding the system under test. You need to be able to configure that system under test to make sure to it's ready to receive messages and when things don't behave as expected, know how to work with that system to determine whether it's actually an error or if it's just misconfigured. Once again, a full description of each of these is in Module T251, but if you're taking this advanced level course, we assume that you have these skills.

Ken Vaughn: So at this point we have downloaded the software and we've installed it on one machine. The next question becomes: How do we actually connect our systems? Well, the C2C RI tool is designed to connect and test center-to-center communications and to make sure that they're conformant with our industry standards. To understand how to use this tool, we first have to understand some of the base terminology concepts involved in C2C communications. With any exchange in the ITS center-to-center communications, one center is called the Owner Center, and the other center is called the External Center. The Owner Center is generally the one providing the ITS information to an external requester. The process used for the two systems to exchange data involves two steps. The first step is to discover the services that the Owner Center provides, and the second is to exchange the data using those services. The discovery is archived by the Owner Center, providing a file that defines the services that it offers using an open format.

Ken Vaughn: Using our deployed environment as the base condition, this graphic depicts how the C2C RI can serve as the Owner Center or the External Center. The C2C RI is configured for one mode and used to test the other mode. So in this case, what's shown on the screen right now, you see the C2C RI acting as an External Center and it could be used to test an Owner Center, but it can be used the other way around. In our particular presentation-- well, the point here is that the C2C RI would replace one end of the system. When you do this, this is simulated testing, that you're actually removing one of the real systems and replacing it with the C2C RI.

Ken Vaughn: Now in our presentation, we're going to be demonstrating both modes of the C2C RI and then we'll use it to test it against itself in the other mode. So we actually have two instances of the software running. This is a useful way to test your setup to ensure that you have the C2C RI configured correctly. Nonetheless, while our demonstrations will test between two C2C RI implementations, realize that a real test would only replace one of the end systems.

Ken Vaughn: With that, we come to our first pop quiz. The question here is: Which of the following statements is untrue, or not true? C2C standards define how services can be discovered; a C2C exchange occurs between an Owner Center and External Center; option C, the C2C RI software is available for free; or option D, ITS standards only define how to exchange information. So go ahead and make your selection and we'll review the answers.

Ken Vaughn: Well, the correct answer is in fact answer D, as in dog. ITS standards also define not only how to exchange information but how to use the information once it's been exchanged. C2C standards do define how services can be discovered. This is done through a WSDL file, which we'll discuss here in a second, and it allows an External Center to discover the services offered by an owner center. The C2C exchange does occur between an Owner Center and an External Center. Those are just names that have been assigned to the two different systems according to the ITS standards. And the C2C RI software is in fact available for free. It can be downloaded from the web address shown here.

Ken Vaughn: Well, to set up the C2C RI, there's a couple of steps involved. One is you have to install the C2C RI on your computer and then define some of the initial configuration of that C2C RI. Once the C2C RI is properly installed, we will then configure it to support the services offered by the Owner Center through the use of a service announcement file. Finally, we'll identify test cases that will be performed in our full deployment through the configuration.

Ken Vaughn: The C2C RI is distributed as a standard Windows installer. When running this installer, it is strongly recommended to install the program into its default directory to avoid any erroneous error warnings when running the software. There are several different directories you have to configure. You can do that after the fact, and we'll show you how to do that, but to minimize any problems, we recommend installing the default directory. Once the software is installed, you'll need to configure the default directories for file storage along with some other basic information.

Ken Vaughn: This video shows that process of setting up some of the initial directories, and you can go ahead and press Play on the video and you'll see how that process actually works in real-time. It is important to remember that all directories must exist on the local machine. Changing the values is especially important if the C2C RI is not installed in the default location. That's why we recommend putting it into the default location, but if the default directories don't exist on the machine, you'll probably get some errors and they may not be completely informative. So just make sure directories are set up correctly.

Ken Vaughn: Now that we've installed the software, we can investigate how to use it. We mentioned that the first part of a connection between two centers is the discovery of services. Services are defined in what's called the Web Services Description Language, or WSDL file. The WSDL file provides a definition of the services and messages supported by the Owner Center. It may also identify publication services that should be supported by External Centers that use subscription service. So in other words, an External Center would access a subscription service, but in order to access the subscription service, the External Center should support a publication service that the Owner Center would then send messages to later on.

Ken Vaughn: Discovery is achieved by the External Center retrieving the WSDL file from the Owner Center, further requirements of NTCIP 2306. It is the responsibility of the Owner Center to define its own WSDL file that conforms to 2306, and to make it available to authorized External Centers. Implementers should be aware that several off-the-shelf tools that assist in the generation of WSDL files do not produce files that are conformant to some of the rules in the current version of 2306. It is the responsibility of the implementer to overcome these obstacles. There's no restriction on the number of WSDL files a center publishes. Thus, an implementation can publish both the generated version and the 2306 version if needed. In short, it is the responsibility of the Owner Center to advertise the services that it offers. It does this through the use of the WSDL file. An External Center only needs to read this file and determine which services it wishes to use. Now the WSDL file format is based on XML. Thus these documents are relatively easy to read, but you have to understand how that structure works. But if you do, you can then use any number of off-the-shelf text editors to edit these documents. The primary purpose of the WSDL, once again, is to define the supported services of the owner system. The primary purpose is to define the supported services to be used in real time. The WSDL file will also be used-- because it's there, it will also be used by the C2C Reference Implementation as part of testing, but the primary purpose is to define the supported services.

Ken Vaughn: The WSDL file defines services in a standardized format. The WSDL file starts with references to the TMDD and other schema files that define the standardized as well as any customized detailed data structures. The file then combines these data structures into messages which are also standardized by the TMDD, and then defines how the messages can be ordered to form dialogs known as operations. Again, this whole process is defined by the TMDD. The WSDL file then associates the operations with the ports and binds them to specific protocol stacks. This is standardized to a small number of options. Finally, the file maps the bindings to services which are assigned a specific URL. The URL is specific to a deployment. So the good news is that most of this information is defined in the standards. If you're using a strictly standardized interface, then you have very little to do, but that last step, defining the URL that is specific to your project. There's no way for the standards to know your specific URL and you'll need to go in and customize that URL to your particular project. Further, if you want to offer services that aren't standardized in addition to the standardized services, you would need to define and customize the rest of the file.

Ken Vaughn: Now WSDL files are associated with the Owner Center. So the Owner Center is responsible for defining the WSDL file and storing it on its website or some location that an External Center can retrieve it. If you're operating the C2C RI in External Center mode, it would just need to configure the C2C RI to reference the WSDL file provided by the Owner Center, i.e., the system under test. In this case, the C2C RI will check the provided WSDL file for conformance to the standards. As mentioned earlier, some deployments have discovered that the WSDL automatically generated by some tools do not conform precisely to the NTCIP 2306 format. The C2C RI will flag these issues so that the developer of the Owner Center can correct them. Now if you're using the C2C RI in Owner Center mode, you'll need to provide the WSDL file for the External Centers to reference. Luckily the C2C RI includes sample WSDL files, but these files will need to be updated to reflect the URL currently in use by the C2C RI Owner Center. For example, you'll need to change the default web address to reflect the IP address or domain name of your C2C RI instance. This is done in the last few lines of the file, and we'll show you a video of that in a second. In fact, that'll be the next slide. It's done for you in one sample file, but be aware that the C2C RI relies upon several different instances of WSDL files for different tests. You'll need to change all of these files that you use in order for your test to work properly.

Ken Vaughn: With that, this video will show you how to change those segments.

Ken Vaughn: So we've now saved and created our WSDL file, but now we need to go ahead and configure the C2C RI to actually use those WSDL files. This video will discuss how you do that, and you can play Record-- or press Play to see how you do that.

Ken Vaughn: Once you've configured the C2C RI to use the appropriate WSDL files, now the next step is to go in and configure the C2C RI to conform to the needs and requirements that you have for your specific project. You should have already done this through a PRL as part of your procurement process, and whatever protocol requirements list you used or needs-to-requirements list that you used to procure your system, this is where you'll enter that information, and it's almost a one-to-one correspondence. This video will show you specifically how you do that.

Ken Vaughn: Now you're fully configured and ready to test. We've discussed how to install, configure the C2C RI, and have customized the WSDL file. We've then configured the C2C RI to use that WSDL file and we configured the C2C RI to use the needs-to-requirements that you've defined for your project. The next step will be actually running tests.

Ken Vaughn: But before we do that, we have our second pop quiz. What is the primary purpose of a WSDL file? A, to configure the C2C RI; B, to report errors to the user; C, define services offered by the Owner Center; or D, all of the above. So go ahead and make your answer.

Ken Vaughn: Well, the answer is C. The WSDL file defines the services offered by the Owner Center and how remote systems can access these services. While it is used for configuring the C2C RI, the purpose of the file is much broader and includes operational deployment. It does not in fact report errors to the user, and because it does not report errors to the user, "All of the above" is also an incorrect option.

Ken Vaughn: Well, that completes our first learning objective, installing and configuring the C2C RI on a PC. The second learning objective is actually operating the C2C RI.

Ken Vaughn: And we'll learn how to actually operate it and understand how to interface with it.

Ken Vaughn: The first video that we'll show here is designed to show you how to actually execute a particular test. So in the first segment, we defined how you configure the C2C RI to reflect the needs and requirements of your particular project. Based on those needs and requirements, the system will automatically identify which tests might be appropriate to your particular project. It is then up to you to identify which tests you want to perform and then actually go through the process of performing those tests. This step will show you how to start a test on an Owner Center, including how you get past all of the initial prompt screens.

Ken Vaughn: The last video described how you configured a system and start a test for the Owner Center. This slide will do the same thing, but it will be for the External Center. Once again, an Owner Center communicates with an External Center. The C2C RI can perform either role. Last slide we talked about the Owner Center; this slide, we'll talk about the External Center.

Ken Vaughn: We now have both an Owner Center and an External Center configured, ready to go; we understand how they both work. This slide will actually show the two centers working side-by-side exchanging data, and it shows you how the system produces results.

Ken Vaughn: Now, you may have already noticed entering variables at the prompts, especially entire XML messages, can become quite time-consuming and prone to error. Any error in those entries can cause a test case to fail and the prompts provide little screen space to double-check what you've entered for the XML message. In order to overcome this, the C2C RI tool allows the user to provide this information in a data file rather than directly in user prompts. This allows the user to enter the data once and then perform the test multiple times. The User Guide describes how to create a data file from scratch; however, really the easiest way to create these files is to go in and edit the default data files that are hidden within the tool itself. We'll investigate how to do that in our next video.

Ken Vaughn: So this video will go in, show you how to obtain a copy of those hidden data files so that then you can go in and simply modify those as needed.

Ken Vaughn: As shown in the video, the tester still needs to know how to read, edit, and create both WSDL and XML files. However, data files allow the tester to prepare this information once for each scenario to be tested and then streamline the testing process over and over again if needed. Within Version 1 of the C2C RI, the response message configured in the Owner Center must be the exact message that you want sent to the External Center as a part of your testing process. However, in Version 2, the XML message sent back-- the configuration file can contain a superset of the information that you want sent back. Version 2 will enable filtering of that message to meet the needs of a particular request. So for example, if the request is for assigned inventory within a certain geographical area, the Owner Center can be configured with a message that contains signs in a larger region, and when that request comes in for that small area, the C2C RI in Version 2 will subset that response message and send back only those signs that are in the requested area. In Version 1, it would send back the entire message, whatever that entire message is. So if you show the entire region, it would send back all of the signs, which isn't really technically true because the message request is asking for that large list to be filtered. Version 2 will include that filtering logic.

Ken Vaughn: It's also worth noting that the C2C RI supports sending multiple publications to a single subscription message. So for example, I can configure my External Center to request to subscribe for information about events. I can then configure my Owner Center copy of the C2C RI to support multiple events reported at different times. So I can have one publication sending out that shows a particular event, and then at some point later a second publication going out to that same original subscription showing a second event. So the data file for a subscription publication test case can be written to define different variable values, so each of those two messages sent from the Owner Center to the External Center can in fact be completely different and the user can define those, and that's done by using this "iteration" tag. So you'd say ITERATION NAME = One, and then you'd give all the parameters you need for that particular iteration, and those are essentially the prompts you would receive if you didn't have a data file. And then if you wanted to have a second publication, you would simply have, in a new line, that says ITERATION-- once again-- NAME = Two, and then you'd give those same variables with whatever values you want. So that allows multiple publications for a single subscription.

Ken Vaughn: Now that brings us to our third pop quiz.

Ken Vaughn: How can you get the C2C RI to publish multiple publications for one subscription? A, enter the number of publications in the configuration file; B, use the #ITERATION keyword in the associated data file; C, multiple publications are not supported by the C2C RI; or D, the C2C RI can receive but not published multiple publications.

Ken Vaughn: Well, the correct answer is B as in boy. The user uses the #ITERATION keyword in the data file. A is incorrect because the configuration file does not have a parameter like this, does not have the number of publications parameter. C is incorrect because the C2C RI does support multiple publications using the #ITERATION keyword. And D is incorrect because the C2C RI is able to both send and receive multiple publications.

Ken Vaughn: Well, that complete Learning Objective 2. We've talked about installing and configuring the C2C RI on a PC, and we've also talked about operating it.

Ken Vaughn: Now we'll talk about retrieving the test results from a test.

Ken Vaughn: The C2C RI presents results in real-time on the screen. The upper pane provides a summary for each test performed and the lower pane shows the step-by-step details of the selected test. When a step fails, the Results cell provides a description of that failure. This allows the user to diagnose the problem without having to go through the entire process of generating a report. The downside is that the screen space is limited and the report format, which we'll discuss later, may offer an easier way to read the information.

Ken Vaughn: It should be noted that when the failures occur, they may be due to a variety of different reasons, as we mentioned earlier. Other than the system under test having an error, there are a variety of different possibilities. While that is one potential source of the problem, the problem could also be due to configuration error, a network configuration error, a user error, a C2C RI configuration error, an implementation error, or an ambiguity in the standard. Each failure reported by the C2C RI must be investigated in detail to determine the cause of the failure. What we don't want to do is to blame the implementation for everything when in fact it may have been just our own user error.

Ken Vaughn: As an example, let's investigate the error reported on the previous screen. This says that the test failed due to a transport error result. In other words, an error occurred related to the transport layer of the communications-- or in our case, the TCP connection. Now, let's consider what the likely sources are for this error.

Ken Vaughn: The TCP stack tends to be off-the-shelf software is used for vertically any communication task. So it's unlikely that the error resides in the implementation of either the SUT or the C2C RI. Further, the standard has been well proven over the years, so it's unlikely that the problem is due to an ambiguity in that standard. Finally, it is unlikely that any user error performing the test would result in such a low-level error. Well, that only leaves three potential errors that are the ones most likely to be the problem. One would be the improper system under test configuration, such as having the system under test listen to the wrong port number, so it never hears the message coming in. Another possibility is the improper network configuration. By this we mean something along the lines of the network is configured in a way that prevents the message from properly being routed. So if I'm going through a complex network, maybe one of my routers is misconfigured. A third possibility is improper C2C RI configuration error, such as sending to the wrong IP address or port number. So any one of those three would be the most likely sources for your problem. This is almost certainly a configuration error rather than an error with the system under test. That's the process you go through for every error identified. Identify what types of errors are unlikely, and then from those you can identify which ones are more likely, and then you can investigate each one of those.

Ken Vaughn: So the user can then diagnose the issue by simply following the logical flow of information. The C2C RI logs can assist in isolating the error. The Message Summary report will identify the messages that are sent and received by the C2C RI. There's a good chance that the system under test will have a similar log capability. So one should be able to check to see if one system attempted to send a message while the intended recipient never device it. However, the final resolution of the problem may need to use a line analyzer to identify why one system believes it is sending a message while the other system does not record it. So for example, maybe the sending system was never able to actually transmit on the wire. Maybe it was using a different communications facility, so it never hit the wire that the other machine was listening to. Or, conversely, you discover that the message did go across that wire. The receiving system software may have received it at a very low level, but for some reason rejected the message because maybe it detected a bit error or something else. The same sort of process of following the logical flow of what should be happening should assist in identifying the source of the problem.

Ken Vaughn: All issues should be documented in anomaly reports. These reports follow the format defined in IEEE 829-2008, and should provide enough information that the issue can be recreated. Once the anomaly is identified, it should be resolved and the system should be retested. A user may wish to test all known failures before starting a retest of the entire system, but as software changes can sometimes have unintended consequences, it is critical to retest all test cases using the final version of the software.

Ken Vaughn: During the test, the C2C RI will automatically record all of the test results. Once the testing session is complete and the user closes the file, the C2C RI will encrypt and sign the information in order to securely store the results. The location of the resultant file can be specified as a part of the directory options under the Tools menu. Of the part of the process, the test is started, or after testing. So the location of the file is either defaulted to the default directory or the user can move it after the test.

Ken Vaughn: That brings us to our fourth pop quiz.

Ken Vaughn: Which of the following might cause the C2C RI to report a failure? An ambiguity in the standard; the system under test configuration error; or the C2C RI configuration error; or D, all of the above.

Ken Vaughn: The correct is D, all of the above. All three options can cause the C2C RI to report a failure. A is incorrect because an ambiguity in the standard can cause a failure, but so can the other options. In the case of B, the SUT configuration error can obviously can obviously cause a test-- so if the system under test is misconfigured, that can also cause an error. And likewise, if the C2C RI is misconfigured, it can also result in an error being reported.

Ken Vaughn: Well, that completes three of our learning objectives. We've installed and configured the C2C RI. We've operated the C2C RI. We then retrieved the test results.

Ken Vaughn: We'll now move on to our fourth learning objective, preparing a report based on the C2C RI test results.

Ken Vaughn: This video will describe how you prepare the reports from within the C2C RI. Now, these are not IEEE 829 reports, but they are the reports that will be generated from the tool that will allow you then to create your own IEEE 829-2008 reports.

Ken Vaughn: It should be noted once again that the C2C RI does not directly generate standardized IEEE 829-2008 reports, primarily because it simply does not have all of the information that it needs. Instead, the C2C RI generates reports that assist users in developing their own IEEE 829 reports. This table maps the IEEE reports to the C2C RI reports that assist in their creation. We'll discuss each of the IEEE 829 reports on the following slides. So you see here Test Log, you might need several different reports because they all log information. The Anomaly Report, same as above; they all want information. Depending on what your error is, you may pull different details from those reports. And then finally the Test Report, there are a few summary reports over in the C2C RI.

Ken Vaughn: The IEEE 829 Test Log is designed to provide a chronological record of relevant details of testing. It should generally log as much information as possible. The C2C RI divides the logs into several distinct reports due to the size these reports can be. The reports include test case details, which is a step-by-step log of the individual steps defined in the test procedure; a message summary, which identifies each message sent or received by the application along with a timestamp of that message; the message details, which shows the complete details of each message sent and received by the application; and then the script log, which provides a log entry every time the test enters or exits a test script. All of these logs can be useful in diagnosing problems and should be incorporated into the final test log. In addition, the IEEE 829 log report may include any logs that are kept separately from the C2C RI. For example, a separate line analyzer tool may provide its own log. Likewise, the tester himself or herself may keep their own written notes that should also be included in the formal IEEE 829 log report.

Ken Vaughn: An Anomaly Report is intended to document any event during the testing process that requires investigation. Any of the C2C RI log reports may be able to assist in identifying these anomalies, but rather than providing the complete log, the anomaly report should analyze that information and provide a concise description of the problem using that log information as necessary. It should also identify the impact of the issue and any corrective action that may be needed. The C2C RI does not provide this level of analysis, only some useful tools to enable the tester to perform that analysis.

Ken Vaughn: Finally, the IEEE 829 defines a test report. This is a summary report that provides evaluation and recommendations for the entire system under test. It should include both the Test Case Summary from the C2C RI, which identifies the pass/fail status of each test as performed in chronological order, and the Conformance Compliance Report, which indicates the pass/fail status for each requirement based on reversing the traceability tables. So for example, if a test fails, the associated requirements will also be marked as failed. Now Version 2 of the C2C RI will also allow the user to generate a Section 1201 conformance report. Section 1201 is the portion of the Safe Accountable Flexible Efficient Transportation Equity Act: A Legacy for Users, more commonly known as SAFETEA-LU. This is the legislation that assures agencies offer real-time monitoring and sharing of travel conditions of major highways across the U.S. The Section 1201 conformance report will show the pass/fail status for each Section 1201 requirement defined in the legislation, and then associate that to the individual tests that were performed.

Ken Vaughn: That brings us to our next pop quiz.

Ken Vaughn: Which C2C RI report will assist in developing an Anomaly Report? Test Case Details; Message Summary; Message Details; or all of the above?

Ken Vaughn: The correct is all of the above. All of those reports can assist in developing an Anomaly Report, but they aren't the Anomaly Report themselves. The Test Case Details report will help identify the step where the error occurs, but other reports can assist as well depending on the type of error. The Message Summary will help identify if messages were sent and received, but once again the other reports help as well. And the Message Detail reports will help identify a message contained an error, make sure it's properly formatted and all of that, but once again, other reports help as well.

Ken Vaughn: Well, that completes our four modules: install and configure the C2C RI on a PC; operating the C2C RI; retrieving the C2C RI results from a test; and preparing a report based on those C2C RI results.

Ken Vaughn: The TMDD testing curriculum is now completed. We started out with the module A321a, which discussed the user needs for the TMDD. A321b talked about the requirements associated with the TMDD. Module T321 talked about applying your test plan in general to the TMDD standard, and Module T251 was an introduction to the tool that we just saw here, the Reference Implementation. And then finally this course, T351, actually stepped you through the process of applying the C2C RI reference implementation.

Ken Vaughn: We thank you for your participation in this course, and we ask for your feedback. You can use the feedback link below to provide us with any of your thoughts and comments about the value of the training. We do value your input and we try to constantly improve our courses. So thanks a lot for your time and we look forward to hearing from you. Thanks.