Module 17 - T311

T311: Applying Your Test Plan to the NTCIP 1203 v03 DMS Standard

HTML of the Student Supplement

(Note: This document has been converted from the Student Supplement to 508-compliant HTML. The formatting has been adjusted for 508 compliance, but all the original text content is included, plus additional text descriptions for the images, photos and/or diagrams have been provided below.)

 

T311 - Applying Your Test Plan to the NTCIP 1203 v03 DMS Standard. See extended text description below.

(Extended Text Description: Large graphic cover page with dark blue background with the title in white letters “T311 - Applying Your Test Plan to the NTCIP 1203 v03 DMS Standard.” At the middle left is the “Standards ITS Training” logo with a white box and letters in blue and green. The words “Student Supplement” and “RITA Intelligent Transportation Systems Joint Program Office” in white lettering are directly underneath the logo. Three light blue lines move diagonally across the middle of the blue background.)

 

T311: Applying Your Test Plan to the NTCIP 1203 v03 DMS Standard

Table of Contents

Purpose  - 2

NTCIP 1203 History - 3

NTCIP 9012 Table of Contents - 4

Presentation Example - 5

Glossary - 12

References - 13

 

PURPOSE

This supplement provides additional information for the Professional Capacity Building (PCB) Module T311, Applying Your Test Plan to the NTCIP 1203 v03 DMS Standard.

Module T311 provides participants with the information needed to assist agencies on how to create a test plan specific to their dynamic message sign (DMS) needs based on the NTCIP 1203 standard.

This module helps the participant understand the elements of the DMS standard that are required to apply test plans to verify that the agency’s DMS system meets the design specifications and is conformant to the NTCIP 1203 standard while following standard testing methodologies. An example is provided in the module.

This module also leverages the NTCIP 9012 v01 standard, Testing Guide for NTCIP Center-to-Field Communications, which provides guidance on:

  1. NTCIP testing concepts and processes;
  2. Where NTCIP testing fits within a broader testing program that includes hardware testing;
  3. Test documentation necessary to ensure that an NTCIP testing program is complete;
  4. A phased approach to test execution;
  5. Testing tools; and
  6. Manufacturer prequalification.

As such, familiarity with the concepts in the NTCIP 9012 v01 standard is a prerequisite for this module.

NTCIP 1203 History


NTCIP 9012 v01.27 Page v

TABLE OF CONTENTS

Section 1 INTRODUCTION - 1

1.1 Overview - 1
1.2 Intended Audience - 1

Section 2 NTCIP AND ITS STANDARDS BACKGROUND - 3

2.1 Purpose of ITS DEVICE Standards - 3
2.2 NTCIP ITS DEVICE Communications Interface Standards - 3
2.3 Hardware and Environmental Standards - 4
2.4 Standards Conformance Versus Agency Specification Compliance - 5
2.4.1 Standards Conformance - 5
2.4.2 Agency Specification Compliance - 6

Section 3 AGENCY REQUIREMENTS AND SPECIFICATIONS - 7

3.1 Testing in the Context of Systems Engineering - 7
3.2 Requirements - 8
3.2.1 ITS Device Communications Interface Requirements - 8
3.2.2 Hardware and Environmental Requirements - 10
3.3 Agency Specifications - 10
3.3.1 What should be in an Agency Specification - 11
3.3.2 Tailoring an NTCIP Standard to Project Requirements - From PRL to PICS - 12

Section 4 TEST DOCUMENTATION - 14

4.1 Overview - 14
4.1.1 ITS Device Standard - 14
4.1.2 Agency Specification - 14
4.1.3 Agency Test Documentation - 14
4.1.4 Test Documentation Standards - 16
4.2 IEEE 829-1998 Test Plan - 16
4.2.1 Test Plan Identifier and Introduction - 16
4.2.2 Test Items and Item Pass/Fail Criteria - 17
4.2.3 Features to be Tested and Features not to be Tested - 17
4.2.4 Approach, Risks and Contingencies - 17
4.2.5 Suspension Criteria and Resumption - 18
4.2.6 Test Tasks, Test Deliverables, and Schedule - 18
4.2.7 Environmental Needs - 19
4.2.8 Responsibilities and Approvals - 19
4.3 IEEE 829-1998 Test Design - 20
4.4 NTCIP 8007 v01 Test Cases and Test Procedures - 20
4.4.1 Test Case and Procedures Overview - 20
4.4.2 Boundary and Error Condition Tests - 21
4.4.3 Test Case and Procedures Documentation - 21
4.5 Test Execution Reports - 22
4.6 Sample Test Documentation Links - 22
4.7 Summary - 23

Section 5 TEST EXECUTION - 24

5.1 Overview - 24
5.2 ITS Device Testing Phases - 24
5.2.1 Prototype Test and Inspection - 25
5.2.2 Design Approval Test - 26
5.2.3 Factory Acceptance Test - 27
5.2.4 Incoming ITS device Test - 27
5.2.5 Site Acceptance Test - 28
5.2.6 Burn-in and Observation Test - 28

© 2008 AASHTO / ITE / NEMA


Presentation Example

The presentation slide uses the functional requirement “Activate Pixel Testing” as an example on how to use the standard to test that a functional requirement has been fulfilled. The full details of this requirement, including the description of the requirement, the design, and the test case and test procedure for this requirement, is provided below.

The functional requirement is:

3.5.3.1.1.2         Activate Pixel Testing

The DMS shall allow a management station to initiate a pixel test.

In the Requirements Traceability Matrix (RTM), this functional requirement traces to the following design:

Requirements Traceability Matrix (RTM)

FR ID

Functional Requirement

Dialog ID

Object ID

Object Name

Additional Specifications

3.5.3

Monitor the Status of

the DMS

3.5.3.1

Perform Diagnostics

3.5.3.1.1

Test Operational Status

of DMS Components

3.5.3.1.1.1

Execute Lamp Testing

4.2.4.1

5.11.2.5.3

lampTestActivation

3.5.3.1.1.2

Activate Pixel Testing

4.2.4.2

5.11.2.4.3

pixelTestActivation

3.5.3.1.1.3

Execute Climate-Control

4.2.4.3

Equipment Testing

5.11.2.3.5.6

dmsClimateCtrlTestActivation

5.11.2.3.5.7

dmsClimateCtrlAbortReason

4.2.4.2  Activating Pixel Testing

The standardized dialog for a management station to command the DMS to activate pixel testing shall be as follows:

  1. The management station shall SET pixelTestActivation.0 to “test.”
  2. The management station shall repeatedly GET pixelTestActivation.0 until it either returns the value of “noTest” or a maximum time-out is reached. If the time-out is reached, the DMS is apparently locked and the management station shall exit the process.
  3. (PostCondition) The following objects will have been updated during the pixel test to reflect current conditions. The management station may GET any of these objects as appropriate.
    1. pixelFailureTableNumRows
    2. any object within the pixelFailureTable

 

5.11.2.4.3          Pixel Test Activation Parameter

pixelTestActivation OBJECT-TYPE
SYNTAX INTEGER {
--other (1), -retired
noTest (2),
test (3),
clearTable (4) }
ACCESS read-write
STATUS mandatory
DESCRIPTION
"<Definition> Indicates the state of the pixel testing. The actual test routine can vary among different manufacturers. The results of the pixel failure test shall be stored in the pixel failure table. The pixel failure table, pixelFailureTableNumRows objects are cleared (both messageDisplay and pixelTest types)when a pixel test is started (test) or a table is cleared (clearTable). Setting the value to test starts the test, meaning this test is executed once. Pixel failures identified by setting this object to test are entered into the pixelTest type of the pixelFailureDetectionType. The sign controller automatically sets the value of this object back to noTest after completion.

<Object Identifier> 1.3.6.1.4.1.1206.4.2.3.9.7.4"
DEFVAL {noTest}
::= { statError 4 }
-- In v02, the enumerated value of “other” is RETIRED to improve
-- interoperability.

In the Requirements to Test Case Traceability Matrix (RTCTM), this functional requirement traces to the following test cases:

Requirement

Test Case

ID

Title

ID | Title

3.5.3

Monitor the Status of the DMS

3.5.3.1

Perform Diagnostics

3.5.3.1.1

Test Operational Status of DMS Components

3.5.3.1.1.1

Execute Lamp Testing

C.3.5.21

Verify Lamp Test with No Errors

C.3.5.22

Verify Lamp Test with Errors

3.5.3.1.1.2 Activate Pixel Testing

C.3.5.1

Pixel Test - No Errors

C.3.5.2

Pixel Test - Errors

3.5.3.1.1.3 | Execute Climate-Control Equipment Testing

C.3.5.3

Climate-Control Equipment Test - No Errors

C.3.5.4

Climate-Control Equipment Test - Errors

 

C.3.5.1  Pixel Test - No Errors

Test Case:
5.1

Title:

Pixel Test - No Errors

Description:

This test case verifies that the DMS executes a pixel test and verifies that there are no failed pixels.

Variables:

Pixel_Test_Time

From manufacturer’s documentation

Message_Display_Test_Time

From manufacturer’s documentation

Pass/Fail Criteria:

The DUT shall pass every verification step included within the test case to pass the test case.

Step

Test Procedure

Results

Additional References

1

CONFIGURE: Determine the maximum period of time that the pixel test should require (based on manufacturer documentation). RECORD this information as:

»Pixel_Test_Time

 

 

2

CONFIGURE: Determine the maximum period of time that the message display pixel test should require (based on manufacturer documentation). RECORD this information as:

»Message_Display_Test_Time

 

 

3

SET-UP: Ensure that all pixels are functioning prior to this test.

 

 

4

SET the following object(s) to the value(s) shown:
 »pixelTestActivation.0 = 'test' (3)

NOTE--Valid enumerated values are defined in Section 5.11.2.4.3 (Pixel Test Activation Parameter).

Pass / Fail
(Section 3.5.3.1.1.2)

Section 4.2.4.2
Step a

5

GET the following object(s):
 »pixelTestActivation.0

Pass / Fail
(RFC 1157)

Section 4.2.4.2
Step b

6

IF the RESPONSE VALUE for pixelTestActivation.0 equals “test” (3), then GOTO Step 5; otherwise, GOTO Step 7.

NOTE--If the RESPONSE VALUE remains at “test” (3) for more than Pixel_Test_Time seconds, this test fails.

 

 

7

GET the following object(s):
 »dmsPixelStatus.1
 »dmsPixelFailureTestRows.0
 »dmsPixelFailureMessageRows.0

Pass / Fail
(Section 3.5.3.1.3.3)

Section 4.2.4.2
Step c

8

VERIFY that the RESPONSE VALUE for dmsPixelFailureTestRows.0 is equal to 0.

Pass / Fail
(Section 3.5.3.1.3.3)

 

9

GET the following object(s):
 »shortErrorStatus.0

Pass / Fail
(Section 3.5.3.1.2)

 

10

VERIFY that the RESPONSE VALUE for shortErrorStatus.0 has bit 5 (pixel error) cleared.

Pass / Fail
(Section 3.5.3.1.2)

 

11

PERFORM the test case labeled “Activate a Message” (C.3.7.6).

Pass / Fail
(Section 3.5.2.3.1)

 

12

DELAY for Message_Display_Test_Time seconds.

 

 

13

GET the following object(s):
»dmsPixelFailureTestRows.0
»dmsPixelFailureMessageRows.0
»pixelFailureTableNumRows.0

Pass / Fail
(Section 3.5.3.1.3.3)

Section 4.2.4.2
Step c

14

VERIFY that the RESPONSE VALUE for dmsPixelFailureMessageRows.0 is equal to 0.

Pass / Fail
(Section 3.5.3.1.3.3)

 

15

VERIFY that the RESPONSE VALUE for pixelFailureTableNumRows.0 is equal to 0.

Pass / Fail
(Section 3.5.3.1.3.3)

 

16

GET the following object(s):
 »shortErrorStatus.0

Pass / Fail
(Section 3.5.3.1.2)

 

17

VERIFY that the RESPONSE VALUE for shortErrorStatus.0 has bit 5 (pixel error) cleared.

Pass / Fail
(Section 3.5.3.1.2)

 

18

PERFORM the test case labeled “Blank the Sign” (C.3.7.15).

Pass / Fail

(Section 3.5.2.3.1)

 

Test Case Results

Tested By:

Date Tested:

Pass / Fail

Test Case Notes:


 

C.3.5.2  Pixel Test - Errors

Test Case: 5.2

Title:

Pixel Test - Errors

Description:

This test case verifies that the DMS executes a pixel test and verifies that there are failed pixels to be detected by unplugging the power or signal to the pixel boards.

Variables:

Pixel_Test_Time

From manufacturer’s documentation

Message_Display_Test_Time

From manufacturer’s documentation

Pass/Fail Criteria:

The DUT shall pass every verification step included within the Test Case to pass the Test Case.

Step

Test Procedure

Results

Additional References

1

CONFIGURE: Determine the maximum period of time that the pixel test should require (based on manufacturer documentation). RECORD this information as:

»Pixel_Test_Time

 

 

2

CONFIGURE: Determine the maximum period of time that the message display pixel test should require (based on manufacturer documentation). RECORD this information as:

»Message_Display_Test_Time

 

 

3

SET-UP: Unplug the power or signal to several pixels to simulate failed pixels to detect within this test procedure.

 

 

4

SET the following object(s) to the value(s) shown:
 »pixelTestActivation.0 = 'test' (3)

NOTE--Valid enumerated values are defined in Section 5.11.2.4.3 (Pixel Test Activation Parameter).

Pass / Fail
(Section 3.5.3.1.1.2)

Section 4.2.4.2
Step a

5

GET the following object(s):
 »pixelTestActivation.0

Pass / Fail
(Section 3.5.3.1.1.2)

Section 4.2.4.2
Step b

6

IF the RESPONSE VALUE for pixelTestActivation.0 equals “test” (3), then GOTO Step 5; otherwise, GOTO Step 7.

NOTE--If the RESPONSE VALUE remains at “test” (3) for more than Pixel_Test_Time seconds, this test fails.

 

 

7

GET the following object(s):
 »shortErrorStatus.0

Pass / Fail
(Section 3.5.3.1.2)

 

8

VERIFY that the RESPONSE VALUE for shortErrorStatus.0 has bit 5 (pixel error) set.

Pass / Fail
(Section 3.5.3.1.2)

 

9

GET the following object(s):
 »vmsSignHeightPixels.0
 »vmsSignWidthPixels.0

Pass / Fail
(Section 3.5.1.2.2.1)

 

10

RECORD the RESPONSE VALUE for vmsSignHeightPixels.0 and vmsSignWidthPixels.0 as:
 »Actual_Height_Pixels
 »Actual_Width_Pixels

 

 

11

Calculate the number of pixels in the sign. RECORD this information as:
 »Total_Pixels

NOTE--In general, the number of pixels in the sign can be determined by multiplying the sign height in pixels by the sign width in pixels. This algorithm is not valid if the pixels on the sign do not form a perfectly rectangular matrix.

 

 

12

Calculate the number of pixel status objects required to be retrieved using the formula: Total_Pixels / 3,200, rounded up to the next integer. RECORD this information as:
 »Num_Pixel_Blocks

 

 

13

GET the following object(s):
 »dmsPixelFailureTestRows.0
 »dmsPixelFailureMessageRows.0

 

 

14

FOR EACH value, N, from 1 to Num_Pixel_Blocks, perform Steps 14.1 through 14.2.

NOTE--For example, if total pixels equal 3201, N shall be assigned a value from 1 to 2.

 

 

14.1

GET the following object(s):
 »dmsPixelStatus.N

Pass / Fail
(Section 3.5.3.1.3.3)

 

14.2

RECORD the RESPONSE VALUE for dmsPixelStatus.N, dmsPixelFailureTestRows.0, dmsPixelFailureMessageRows.0 as:
»Pixel_Status[N]
»Pixel_Failure_Test_Rows
»Pixel_Failure_Message_Rows

 

 

15

VERIFY that the number of bits set in all of the Pixel_Status[N] parameters equals Pixel_Failure_Test_Rows.

Pass / Fail
(Section 3.5.3.1.3.3)

 

16

FOR EACH value, N, from 1 to Pixel_Failure_Test_Rows, performs Steps 16.1 through 16.7.

 

 

16.1

GET the following object(s):
 »pixelFailureXLocation.2.N
 »pixelFailureYLocation.2.N
 »pixelFailureStatus.2.N

Pass / Fail
(Section 3.5.3.1.4.3)

Section 4.2.4.6
Step c

16.2

Determine the X and Y location of the pixel. RECORD this information as:
 »X
 »Y

 

 

16.3

Calculate the text string describing the location of the failed pixel. RECORD this information as:
 »Failed_Pixel_Location

 

 

16.4

VERIFY that the RESPONSE VALUE for pixelFailureXLocation.2.N and pixelFailureYLocation.2.N identify one of the failed pixels that have not been previously identified.

Pass / Fail
(Section 3.5.3.1.4.3)

 

16.5

VERIFY that the RESPONSE VALUE for pixelFailureStatus.2.N is not equal to 0.

Pass / Fail
(Section 3.5.3.1.4.3)

 

16.6

Calculate the unique pixel number of the failed pixel. RECORD this information as:
 »Subject_Pixel

NOTE--The unique pixel number is defined by its position on the sign, where the top and left-most pixel is pixel 0, the next one to the right is pixel 1, etc. Assuming a perfectly rectangular matrix sign, the pixel number can be calculated by multiplying the (Y position minus 1) by the sign width in pixels and adding the X position minus 1.

 

 

16.7

VERIFY that the bit corresponding to the Subject_Pixel in Pixel_Status[N] is set to one.

Pass / Fail
(Section 3.5.3.1.4.3)

 

17

PERFORM the test case labeled 'Activate a Message' (C.3.7.6).

NOTE--This step is allowed to fail if the sign has internal logic to prevent the display of the message due to an excessive number of pixel failures.

Pass / Fail

(Section 3.5.2.3.1)

 

18

VERIFY that the disconnected pixels are blank.

Pass / Fail
(Section 3.5.3.1.3.3)

 

19

DELAY for Message_Display_Test_Time seconds.

 

 

20

GET the following object(s):
 »shortErrorStatus.0

Pass / Fail
(Section 3.5.3.1.2)

 

21

VERIFY that the RESPONSE VALUE for shortErrorStatus.0 has bit 5 (pixel error) set.

Pass / Fail
(Section 3.5.3.1.2)

 

22

GET the following object(s):
 »dmsPixelFailureTestRows.0
 »dmsPixelFailureMessageRows.0

Pass / Fail
(Section 3.5.3.1.3.3)

 

23

Determine the RESPONSE VALUE. RECORD this information as:
 »Pixel_Failure_Test_Rows
 »Pixel_Failure_Message_Rows

 

 

24

FOR EACH value, N, from 1 to Pixel_Failure_Message_Rows, performs Steps 24.1 through 24.7.

 

 

24.1

GET the following object(s):
 »pixelFailureXLocation.3.N
 »pixelFailureYLocation.3.N
 »pixelFailureStatus.3.N

Pass / Fail
(Section 3.5.3.1.4.3)

Section 4.2.4.6
Step c

24.2

Determine the X and Y location of the reported pixel failure. RECORD this information as:
 »X
 »Y

 

 

24.3

Calculate the text string describing the location of the failed pixel. RECORD this information as:
 »Failed_Pixel_Location

 

 

24.4

VERIFY that the RESPONSE VALUE for pixelFailureXLocation.3.N and pixelFailureYLocation.3.N identify one of the failed pixels that have not been previously identified in the message.

Pass / Fail
(Section 3.5.3.1.4.3)

 

24.5

VERIFY that the RESPONSE VALUE for pixelFailureStatus.3.N is not equal to 0.

Pass / Fail
(Section 3.5.3.1.4.3)

 

24.6

Calculate the unique pixel number of the failed pixel. RECORD this information as:
 »Subject_Pixel

NOTE--The unique pixel number is defined by its position on the sign, where the top and left-most pixel is pixel 1, the next one to the right is pixel 2, etc. Assuming a perfectly rectangular matrix sign, the pixel number can be calculated by multiplying the (Y position minus 1) by the sign width in pixels and adding the X position minus 1.

 

 

24.7

VERIFY that the bit corresponding to the Subject_Pixel in Pixel_Status[N] is set to one.

Pass / Fail
(Section 3.5.3.1.4.3)

 

25

SET-UP: Reconnect the power or signal to the pixels from which it was removed.

 

 

26

GET the following object(s):
 »shortErrorStatus.0

Pass / Fail
(Section 3.5.3.1.2)

 

27

VERIFY that the RESPONSE VALUE for shortErrorStatus.0 has bit 5 (pixel error) cleared.

Pass / Fail
(Section 3.5.3.1.2)

 

28

PERFORM the test case labeled 'Blank the Sign' (C.3.7.15).

Pass / Fail

(Section 3.5.2.3.1)

 

Test Case Results

Tested By:

Date Tested:

Pass / Fail

Test Case Notes:

 

Glossary

The following is a glossary of terms that are used throughout the module.

Term

Definition

Compliance

A condition that exists when an item meets all of the requirements of an agency specification.

Device Under Test (DUT)

NTCIP device that is the object of testing.

Requirements To Test Cases Traceability Matrix (RTCTM)

A table that defines the traceability from a requirement to the associated test case.

Requirements Traceability Matrix (RTM)

A table that links the requirements to the corresponding dialogs and objects.


 

References

Dynamic Message Signs

Systems Engineering

Testing