Module 19 - C101
C101: Introduction to Communications Protocols and their Uses in ITS Applications
HTML of the Student Supplement
(Note: This document has been converted from the Student Supplement to 508-compliant HTML. The formatting has been adjusted for 508 compliance, but all the original text content is included, plus additional text descriptions for the images, photos and/or diagrams have been provided below.)
(Extended Text Description: Large graphic cover page with dark blue background with the title in white letters “C101: Introduction to Communications Protocols and their Uses in ITS Applications.” At the middle left is the “Standards ITS Training” logo with a white box and letters in blue and green. The words “Student Supplement” and “RITA Intelligent Transportation Systems Joint Program Office” in white lettering are directly underneath the logo. Three light blue lines move diagonally across the middle of the blue background.)
C101: Introduction to Communications Protocols and their Uses in ITS Applications
Table of Contents
Introduction/Purpose - 2
NTCIP Framework - 3
List of NTCIP Device Standards (C2F) that Use SNMP - 6
Glossary - 6
Center-to-Center Terminology - 10
References - 13
Study Questions - 14
This module is a part of the acquisition curriculum path with I101, A101, C101, C201, and C202. The course is designed to provide an overview of the communications process as it is used in the NTCIP Framework. Upon taking this course, a student should be able to understand how the remote management of ITS field devices from a central management station works and how various devices can share the same system environment and be compatible. This will prepare students in understanding device standards used in ITS applications and the testing phase of system acceptance. This module forms the basis for discussion on Simple Network Management Protocol (SNMP) as used in NTCIP Center-to-Field (C2F) Framework in module C201: Introduction to Simple Network Management Protocol and its Applications in the Field Devices Based on the NTCIP Standards and application level profile standard NTCIP 2306 XML discussed in module C202: Introduction to the Application Level Protocols for Center-to-Center Communications Systems Interface Implementation. The combination is shown below:
- C101 and C201 prepares students for all A3xx device standards modules and related testing modules (T3xx) designed for center-to-field communications (C2F)
- C101 and C202 prepares students for A321a, A321b and T321 Modules designed for center-to-center communications (C2C)
Both C101 and C202 modules contents are based on the concepts provided by the C101 module. The C201 module expands on the C2F communications using SNMP interface with the field devices such as Dynamic Message Sign (DMS), Closed-circuit TV (CCTV) and Environmental Senor System (ESS). The C202 module deals with how centers communicate with each other in real-time to exchange information and use that information for management of traffic and emergencies.
The National Transportation Communications for ITS Protocol (NTCIP) family of standards offers two categories of standards for use in data communications standards. When used for remote control of the roadside and other transportation management devices, NTCIP-based devices and software can help achieve interoperability and interchangeability. When used between transportation and emergency management centers, NTCIP standards facilitate agency coordination and information sharing.
This module introduces basic concepts of the International Organization of Standards (ISO) seven layers Open Systems Interconnection Reference Model (OSI-RM), and mapping to the five levels of NTCIP Frameworks, which contain protocols for center-to-field (C2F) and center-to-center (C2C) communication. Modules explain the application of these protocols in deploying field devices such as DMS, CCTV, and ASC without going into details on protocols constructs. In the transportation field, we are concerned with the concepts of compatibility, interoperability, and interchangeability and how to achieve these objectives with NTCIP-based deployments.
NTCIP defines a family of general-purpose communications protocols and transportation-specific data dictionaries/message sets that support most types of computer systems and field devices used in transportation management. Applications for NTCIP are generally divided into two categories: C2F and C2C. The former, C2F, normally involves devices at the roadside, communicating with management software on a central computer. The latter, C2C, usually involves computer-to-computer communications where the computers can be in the same room, in management centers operated by adjacent agencies, or across the country. For both C2F and C2C applications, NTCIP supports systems and devices used in traffic, transit, emergency management, traveler information, and planning (data archiving) systems.
2. NTCIP Framework
The seven layers in the NTCIP framework are somewhat different from communication stack layers defined by ISO's Open Systems Interconnection (OSI) seven-layer Reference Model and other standards developing organizations. The OSI model breaks the communications process into seven well-defined layers. Each layer has a defined purpose, generally independent of adjacent layers. Although OSI communications protocols are not widely used, the layered model remains.
(Extended Text Description: Relevant description from author's notes: NTCIP Framework: A graphic of the communication levels of the NTCIP standards. The bottom level is the Plant Level and includes boxes for Dial-up, Fiber, Coax, Wireless, Twisted Pair, and Leased Line. The next higher level is called the Subnetwork Level and includes PPP, Ethernet, and PMPP. The next level is called the Transport Level and includes TCP/IP, UDP/IP, and T2/NULL. The next level is called the Application Level and includes C2C XML, DATEX, FTP, TFTP, SNMP, and STMP. The next level is called the Information Level and includes C2C Messages, Files, Data Objects, and Dynamic Objects. These boxes are connected to an overarching box also in the Information Level labeled Functional Area Data Dictionaries with the left hand side identifying C2C Data Dictionaries and the right hand side labeled NTCIP Data Dictionaries. )
Figure 1: NTCIP Framework (Source: NTCIP Guide)
The NTCIP Framework as shown in Figure 1 above extends beyond the communications OSI RM stack to include informational data and interfaces to the physical communications infrastructure. The levels and terminology used in NTCIP were chosen for simplicity and ease of understanding by lay readers, and relevance to typical applications in the transportation industry. The OSI layers and terminology are often referenced in later technical sections of this publication and in many of the standards defined by NTCIP. The NTCIP framework shown above contains NTCIP Information, Application, Transport, Subnetwork, and Plant Levels loosely related to the OSI model.
Where are data standards located?
NTCIP Information Level—Information standards define the meaning of data and messages and generally deal with ITS information (rather than information about the communications network). This is similar to defining a dictionary and phrase list within a language. These standards are above the traditional ISO seven-layer model. Information level standards represent the functionality of the system to be implemented.
Where are Protocol Standards Located?
NTCIP Application Level—Application standards define the rules and procedures for exchanging information data. The rules may include definitions of proper grammar and syntax of a single statement, as well as the sequence of allowed statements. This is similar to combining words and phrases to form a sentence, or a complete thought, and defining the rules for greeting each other and exchanging information. These standards are roughly equivalent to the Session, Presentation, and Application Layers of the OSI model.
Where are Transport Protocol Standards Located?
NTCIP Transport Level—Transport standards define the rules and procedures for exchanging the application data between point "A" and point "X" on a network, including any necessary routing, message disassembly/re-assembly, and network management functions. This is similar to the rules and procedures used by the telephone company to connect two remotely located telephones. Transportation level standards are roughly equivalent to the Transport and Network Layers of the OSI model.
Where are Subnetwork Profile Standards Located?
NTCIP Subnetwork Level—Subnetwork standards define the rules and procedures for exchanging data between two "adjacent" devices over some communications media. This is equivalent to the rules used by the telephone company to exchange data over a cellular link versus the rules used to exchange data over a twisted pair copper wire. These standards are roughly equivalent to the Data Link and Physical Layers of the OSI model.
How does the Lower Layer Function in NTCIP?
NTCIP Plant Level—The Plant Level is shown in the NTCIP Framework only as a means of providing a point of reference to those learning about NTCIP. The Plant Level includes the communications infrastructure over which NTCIP communications standards are to be used and has a direct impact on the selection of an appropriate Subnetwork Level for use over the selected communications infrastructure. The NTCIP standards do not prescribe any one media type over another. In most cases, communications media selections are made early in the design phase.
What is a Profile standard?
Profile: A profile standard combines one or more base standards and selects appropriate options or functions within them. (A base standard may be a "standard" or another profile that references standards.)
Devices that use any particular subnetwork protocol can share the same communications line with other devices using the same subnetwork protocol. It doesn't matter whether such devices are from different manufacturers or are totally different devices; for example, a traffic signal and a dynamic message sign. Each device is assigned an address that is unique on that line or channel.
- Ethernet: This subnetwork profile specifies the provisions for a connectionless and connection-oriented data link service and the physical interface between an end system and other compatible end systems.
- Point-to-Point Protocol (PPP): Point-to-Point Protocol (PPP) is a protocol that operates in a point-to-point configuration where exactly two devices (called peers) are connected by a communications link. PPP is intended to provide an interoperability standard for transportation related devices for dialed-up circuits using V Series Modems.
- Point-to-Multipoint Protocol (PMPP): Point-to-Multipoint Protocol (PMPP) is a protocol that operates in a primary/secondary configuration where one device is the designated primary, while one or more other devices are connected to one communication channel acting as secondary. PMPP is intended to provide an interoperability standard for transportation related devices using frequency shift keying (FSK) modems.
C2F Communications Stack Example
- A C2F stack is created by choosing the protocols at each level .
- Select device data standards (12xx)
- Select a protocol (23xx) (SNMP)
- Non-routing, no transport profile needed
- Routing-TCP/IP or UDP/IP for SNMP (2202)
- PMPP (2101-2102)
- Leased Line or Fiber (example)
3. List of NTCIP Device Standards (C2F) that Use SNMP
- NTCIP 1201 v03 Global Objects (GO) Definitions (Companion standard for all device standards listed below)
- NTCIP 1202 v02 Object Definitions for Actuated Traffic Signal Controller (ASC) [ASC can also use STMP]
- NTCIP 1203 v03 Object Definitions for Dynamic Message Signs (DMS)
- NTCIP 1204 v02 Environmental Sensor Station (ESS) Interface Protocol (v03)
- NTCIP 1205 v01 Object Definitions for Closed Circuit Television (CCTV) Camera Control
- NTCIP 1206 v1.23 Object Definitions for Data Collection and Monitoring (DCM) Devices
- NTCIP 1207 v02 Object Definitions for Ramp Meter Control (RMC) Units
- NTCIP 1208 v1.12 Object Definitions for Closed Circuit Television (CCTV) Switching
- NTCIP 1209 v02 Data Element Definitions for Transportation Sensor Systems
- NTCIP 1210 v1.53 Field Management Stations - Part 1: Object Definitions for Signal System Masters
- NTCIP 1211 v01 Object Definitions for Signal Control and Prioritization
- NTCIP 1212 NTCIP Objects for Network Camera Operation (Work Pending)
- NTCIP 1213 v02 Object Definitions for Electrical and Lighting Management Systems (ELMS)
(Reference: www.ntcip.org. Please note that the version numbers may have changed during the documentation updating process.)
Communications: Information transfer among users or processors according to agreed conventions and a branch of technology concerned with the representation, transfer, interpretation and processing of data among persons, places and machines. Further, the meaning assigned to the data must be preserved during these operations.
Data Dictionaries: An organized and constructed (electronic database) compilation of descriptions of data concepts that provides a consistent means for documenting, storing, and retrieving the syntactical form (i.e., representational form) and the meaning and connotation of each data concept (ISO 14817).
Message: A grouping of data elements that encapsulate an idea, concept or thing, or convey information. A basic message encapsulates an idea, concept or thing, and a compound message embeds one or more basic messages and other data elements to convey information.
Dialog: An ordered grouping of messages exchanged between at least two components.
Compatibility: "The ability of two or more systems or components to perform their required functions while sharing the same hardware or software environment." Ref. IEEE 610 Std. (Ref.4)
Interoperability: IEEE Standard Glossary of Software Engineering Terminology (IEEE 610 standard, Ref.4) defines interoperability as the ability of two or more systems or components to exchange information and to use the information that has been exchanged.
Interchangeability: A condition which exists when two or more items possess such functional and physical characteristics as to be equivalent in performance and durability, and are capable of being exchanged one for the other without alteration of the items themselves, or adjoining items, except for adjustment, and without selection for fit and performance. (Ref. National Telecommunications and Information Administration, U.S. Department of Commerce)
Management System: The technology used to manage a network. Usually Network Management Station (NMS) is referring to the management of networking specific devices such as routers, or in NTCIP devices in the field. In the context of the NTCIP, NMS refers to all devices including end systems that are present on the network or inter network. Figure 2 introduces an SNMP model used in the NTCIP C2F communications.
(Extended Text Description: SNMP Model: The figure conveys that the SNMP Model consists of SNMP Manager and SNMP Agent. This is shown in two large boxes. The box one on left is called a Management System. In this box with light boundary lie three sub-boxes: a square box is SNMP Manager, an arrow from this leads to a cylindrical shape marked as a MIB. A rectangular box is called Central Application system is shown below SNMP Manager. Together, these three parts are called Management Station. The large box on the Right is called “Managed device”. In that box, lie three parts: SNMP Agent, MIB and Device firmware. A one way arrow from SNMP is connecting to MIB, a two way arrow from ANMP Agent connects to Device Firmware. Together, these parts work with each other inside the device being managed. There are three one way arrows shown from the first large box-SNMP Manager connecting to SNMP Agent. This implies three commands performed by SNMP Manager. There are two one way arrows from SNMP Agent to SNMP Manager are shown-which implies two commands an agent performs.)
Figure 2: SNMP Model
SNMP Network Model Key Parts
- SNMP Manager: An application program that contacts an SNMP agent to query or modify the database at the agent.
- SNMP Agent: Software that runs on a device and maintains information about configuration and current state of database. Figure 3 context diagram shows how the agent process connects to device internals. (Readers may refer to the MIB text book by David Perkins-Ref 1 for greater details on how an agent process works).
- SNMP protocol: The application layer protocol used by SNMP agents and managers to send and receive data.
- MIB: (Management Information Base) describes the information about the device being managed. An MIB specifies the managed objects. MIB is a text file that describes managed objects using the syntax of ASN.1 (Abstract Syntax Notation 1). ASN.1 is a formal language for describing data and its properties.
SNMP Agent Process
Agent Process Context Diagram
Instrumentation Routines are part of the agent, check if the object is in the MIB, verifies access, and knows where the object is located (and retrieve or set its value)
(Extended Text Description: This figure has both text and block diagram shown to convey how an SNMP Agent in the device works. Agent is software module that receives messages from the SNMP Manager though UDP transport stack and processes the messages. UDP is shown in square box that connects to a circle which is SNMP Agent process. A two way arrow connects MIB (shown as cylindrical shape) to this Agent process. A two way arrow connects SNMP agent process to Instrumentation Routines and device internals. This is called access mechanism. )
Figure 3: SNMP Agent Process Context Diagram
OID: Object Identifier is the unique name (identifier) that is associated with each type of data element in an MIB. This is a defined ASN.1 type. "A value (distinguishable from other such values) that is associated with an object identifier type. A simple type whose distinguished values are the set of all object identifiers allocated in accordance with the rules of [ASN.1]." The number or address by which a data element may be located on the NTCIP or TCIP object tree. The OID generation is shown in Figure 5.
Protocol Data Unit (PDU): PDU is a part of transmitted data that contains information used by the protocol at a particular layer in the OSI stack. SNMP has three outbound PDUs to agent and two PDUs from agent to the SNMP manager.
Port Number: Identifies an application-entry to a transport service in the Internet suite of protocols. The concept of ports is often present in OSI literature; however, ports are not Internet standards, but exist as local network conventions only.
How Object Identifier (OID) is Derived?
OID is an unique number derived from the Global (ISO) tree
(Extended Text Description: Numeric display shows a decimal separated string of digits: 22.214.171.124.4.1.1126.96.36.199.1.1. Each digit corresponds to decimal separated nominal display phrase as: iso.org.dod.internet.priavte.enterprise.nema.transportaion.devices. asc.phase.maxphase. Both are shown connect to each other with slanted lines. The meaning of this diagram is that object OID is generated using the Internet tree-structure. )
Figure 4: Object Identification Tree Structure
VarBind and VarBindList (Variable Binding)
varBind and VarBindList
- VarBind is a pairing of object instance (also called variable) name (OID) with an associated value
- VarBindList is a sequence of number of varBinds; they form !'payload': in message PDU
(Extended Text Description: This figure shows rectangular array; VarBind 1 is placed in a box with two small sub-boxes: name and value. Both are shown with green shading. Next box repeats as VarBind 2 and dotted line shows last box as Var-bind n. This conveys that a VarBind list is made of series of VarBinds. The term VarBind means: Name and value of a variable form a binding and move as pair payload or data message PDU. )
Figure 5: VarBind and VarBind List
5. Center-to-Center Terminology
- W3C World Wide Web Consortium
- XML Extensible Markup Language (Encoding method for messages)
- WSDL Web Services Description Language (Centers Public Interface Format)
- SOAP Simple Object Access Protocol (Communication method for XML Messages)
- HTTP Hypertext Transfer Protocol (Web browser Transport Protocol)
- PRL Profile Requirements List (in NTCIP 2306)
- NTCIP 2304 AP: Data Exchange Application Profile standard
- NTCIP 2306 AP: XML Application Profile standard
System Interface (SI): The IEEE Standard Glossary of Software Engineering Terminology (IEEE 610 Standard) defines an interface as a shared boundary across which information is passed. A shared boundary is integrated with the local system applications and termed as a system interface. Typically, an SI can be integrated with the Application Programming Interface (API) of the Advanced Traffic Management System (ATMS), allowing communication to and from the Transportation Management Center (TMC) or an application. We can see in Figure 6 how operation is conducted as input/output messages though a system interface built with the Traffic Management Data Dictionary (TMDD).
(Extended Text Description: The figure shows a big rectangle in which a circle is shown marked as Center System. This circle is separated by a solid line. Next are two small circles each marked as OP1 and OP2. Both have one way arrow coming in as Message Input and one way arrow going out as Message Output. Underneath the two OP circle is the text box for System Interface. This conveys that the operations performed by the system interface use two messages, one incoming-one outgoing. This process is called operation. )
Figure 6: System Interface Operations
Web service: A Web service is traditionally defined by the W3C as "a software system designed to support interoperable machine-to-machine interaction over a network." It has an interface described in a machine-processable format (specifically Web Services Description Language WSDL). Other systems interact with the Web service in a manner prescribed by its description using SOAP messages, typically conveyed using HTTP with an XML serialization in conjunction with other Web-related standards. http://en.wikipedia.org/wiki/Web service - cite note-0 (http://www.w3.org/TR/ws-gloss/).
The Web service concept is now part of NTCIP 2306 XML Profile (It is not part of NTCIP 2304 DATEX Profile, which does not use the Internet as a network). A Web service is any service (operation equivalent to functions) that is available over the Internet or on Intranet. A Web service is between two or more applications, and it is called machine-centric for that reason.
- Uses a standardized XML messaging system
- Independent of operating system or programming language
SOAP: A lightweight (simple) XML-based communication protocol for exchanging structured information between distributed applications over native Web protocols, such as HTTP. SOAP specifies the format that XML messages should use (in the case of ITS by referring the XML schemas of each standard); the way in which they should be processed; a set of encoding rules for standard and application-defined data types; and a convention for representing remote procedure calls and responses.
NTCIP 2306 XML Application Profile
- Data Concepts in XML Format: Dialogs-messages-data frames-data elements
- NTCIP 1104 Naming Conventions
NTCIP 2306 XML Profile Standard:
- XML Schema-WSDL-SOAP (messages, interface, and protocol)
- Transport: HTTP or XML
Transmission Control Protocol/Internet Protocol (TCP/IP) (Used in C2C)
- TCP/IP is made up of two protocols, Transmission Control Protocol (TCP) that deals with applications and Internet Protocol (IP) that deals with networks. TCP/IP is the most widely used connection-based protocol for Internet communications. TCP/IP is used for routed networks that require a reliable protocol.
- A reliable protocol, in this context, means that the protocol attempts to detect and recover from transmission errors. The additional reliability also results in reduced efficiency due to overhead within the packet and more processing required.
- Note that the SNMP performs error handling at the application level, making UDP/IP sufficient for most NTCIP applications that use routed networks.
User Datagram Protocol/Internet Protocol (UDP/IP) Used in C2F
- UDP/IP suit consist of two protocols, User Datagram Protocol (UDP) and the Internet Protocol (IP). The connection-less UDP/IP is used for routed networks that do not require a reliable protocol (also known as non-reliable).
- A non-reliable protocol, in this context, means that the protocol does not make any attempt to detect or recover from transmission errors. Any detection and error recovery should be done at a higher layer. Because of this, UDP/IP communications are more efficient than TCP/IP due to reduced overhead and processing requirements. UDP/IP is recommended for NTCIP in routed networks unless the application explicitly requires a connection-based TCP/IP.
- Systems Engineering for ITS-An Introduction for Transportation Professionals, FHWA: http://ops.fhwa.dot.gov/publications/seitsguide/seguide.pdf
- IEEE Standard Glossary of Software Engineering Terminology, IEEE Std. 610.121990.
Papers, Reports on ITS Standards
- An Overview of ITS Standards and Protocols by Raman K. Patel and Edwin Rowe. The Institute of Transportation Engineers. Available online at: www.ite.org/standards/ITS stdp.asp#Important.
- The TCP/IP Guide: Charles Kozierok; Free Access at www.tcpipguide.com/free/t FundamentalNetworkCharacteristics.htm
Published Guides on NTCIP Family and Other ITS Standards Information
- NTCIP Guide, Information Report 9001, www.ntcip.org, NEMA.
- NEMA, www.ntcip.org [Library of standards, publically available for one time download].
- ITE PCB Modules series 100, 200, and 300 Modules [Online] - www.pcb.its.dot.gov/standardstraining/Modules.aspx
- USDOT, RITA, ITS-JPO (Fact sheets), www.its.dot.gov/index.htm
- USDOT Standards Program, www.standards.its.dot.gov
7. Study Questions
Question 1: Center-to-Field (C2F) Device Data Standards are located at:
- Information Level
- Application Level
- Transport Level
- Subnetwork Level
Question 2: NTCIP 12xx Device Standards Provide:
- Management Information Base (MIB) for each field device
- Application protocols such as SNMP and STMP
Question 3: To gather data from a detector station, the central SNMP Manager initiates:
- GetRequest message
- SetRequest message
- Trap message
- GetResponse message
Question 4: Which of the following protocols is used for monitoring a DMS?
- NTCIP 2306 XML
Question 5: Which of the following is NOT an applicable standard to C2C?
- NTCIP 2306 XML Profile standard