Module 18 - T313

T313: Applying Your Test Plan to the NTCIP 1204 v03 ESS Standard

HTML of the PowerPoint Presentation

(Note: This document has been converted from a PowerPoint presentation to 508-compliant HTML. The formatting has been adjusted for 508 compliance, but all the original text content is included, plus additional text descriptions for the images, photos and/or diagrams have been provided below.)

 

Slide 1:

Slide 1: ITS Welcome - see the extended text description below.

(Extended Text Description: Slide 1: Welcome - Graphic image of introductory slide. A large dark blue rectangle with a wide, light grid pattern at the top half and bands of dark and lighter blue bands below. There is a white square ITS logo box with words “Standards ITS Training” in green and blue on the middle left side. The word “Welcome” in white is to the right of the logo. Under the logo box are the words “RITA Intelligent Transportation Systems Joint Program Office.”)

 

Slide 2:

Welcome

Head shot photo of Shelley Row, P.E., PTOE - Director - ITS Joint Program Office

Shelley Row, P.E., PTOE

Director

ITS Joint Program Office

Shelley.Row@dot.gov

Screen capture snapshot of RITA website - for illustration only - see the extended text description below.

(Extended Text Description: Slide 2: Screen capture snapshot of RITA website - for illustration only. Below this image is a link to the current website: http://www.pcb.its.dot.gov - this screen capture snapshot shows an example from the RITA website from June 3, 2011. At the top of the page it shows the RITA logo with the text Research and Innovative Technology Administration - Intelligent Transportation Systems. Below the main site banner, it shows the main navigation menu with the following items: About RITA, Communities of Interest, Contact Us, Press Room, RITA Offices, Site Map, and a Search button. Below the main navigation menu, it shows a sub-navigation menu with the following items: About Us, T3 Webinars, ITS Peer-to-Peer, Resources, Local ITS PCB and Testimonials. Beneath the sub-navigation menu, the page is sub-titled "ITS Professional Capacity Building Program" and is divided into sub-sections such as "Welcome to ITS Professional Building", "News", "ITS Technical Assistance" and "Scheduled T3 Webinars". Again, this image serves for illustration only. The current website link is: http://www.pcb.its.dot.gov)

WWW.PCB.ITS.DOT.GOV

(Note: There is additional text attached to this slide that includes the following introductory information from Shelley Row):

"ITS Standards can make your life easier. Your procurements will go more smoothly and you’ll encourage competition, but only if you know how to write them into your specifications and test them. This module is one in a series that covers practical applications for acquiring and testing standards-based ITS systems.

I am Shelley Row the director of the ITS Joint Program Office for USDOT and I want to welcome you to our newly redesigned ITS standards training program of which this module is a part. We are pleased to be working with our partner, the Institute of Transportation Engineers, to deliver this new approach to training that combines web based modules with instructor interaction to bring the latest in ITS learning to busy professionals like you.

This combined approach allows interested professionals to schedule training at your convenience, without the need to travel. After you complete this training, we hope that you will tell colleagues and customers about the latest ITS standards and encourage them to take advantage of the archived version of the webinars.

ITS Standards training is one of the first offerings of our updated Professional Capacity Training Program. Through the PCB program we prepare professionals to adopt proven and emerging  ITS technologies that will make surface transportation safer, smarter and greener which improves livability for us all. You can find information on additional modules and training programs on our web site www.pcb.its.dot.gov.

Please help us make even more improvements to our training modules through the evaluation process. We look forward to hearing your comments. Thank you for participating and we hope you find this module helpful."

 

Slide 3:

T313

Applying Your Test Plan to the NTCIP 1204 v03 ESS Standard

 

Slide 4:

Target Audience

 

Slide 5:

Instructor

Photo of the instructor, Russ Brookshire

Russ Brookshire

Product Manager Intelligent Devices Suwanee, GA, USA

 

Slide 6:

Recommended Prerequisites

 

Slide 7:

Curriculum Path (Testing)

Curriculum Path (Testing). Please see the Extended Text Description below.

(Extended Text Description: Curriculum Path (Testing). A graphical illustration indicating the sequence of training modules for the standards that include Testing content.  Each module is represented by a box with the name of the module in it and an arrow showing the logical flow of the modules.  The current module is highlighted. The first box is labeled “T101 Introduction to ITS Standards Testing.”  An arrow from this box connects it to a box labeled “T201 How to Write a Test Plan”.  An arrow from this box connects it to a box labeled “T202 Overview of Test Design Specifications, Test Cases, and Test Procedures”.  Three arrows from this box connect to three additional boxes labeled “T311 Applying Your Test Plan to the NTCIP 1203 v03 DMS Standard”, “T313 Applying Your Test Plan to the NTCIP 1204 v03 ESS Standard” (this module, highlighted), and “T3XX Applying Your Test Plan to NTCIP/TMDD/ATC Standards”.)

 

Slide 8:

Learning Objectives

  1. Describe, within the context of the testing lifecycle, the role of test plans and the testing to be undertaken.
  2. Identify key elements of NTCIP 1204 relevant to the test plan.
  3. Describe the application of a good test plan to an ESS system being procured.
  4. Describe a process of adapting a test plan based on the selected user needs and requirements

 

Slide 9:

Activity. A placeholder graphic with an image of hand over a computer keyboard to show that an activity is taking place.

 

Slide 10:

Environmental Sensor Station

Why Would an Agency Need to Remotely Monitor Environmental Conditions?

Enter response in the chat pod

 

Slide 11:

Environmental Sensor Station

May remotely monitor:

Photo of a typical Environmental Sensor Station. Please see the Extended Text Description below.

(Extended Text Description: Photo of a typical Environmental Sensor Station, including a temperature/humidity sensor, a traffic sensor, two cameras, a control cabinet, and a technician in cold-weather gear.)

Courtesy: Intelligent Devices, Inc.

 

Slide 12:

Learning Objective #1

The Project Lifecycle

The Project Lifecycle. Please see the Extended Text Description below.

(Extended Text Description: The Project Lifecycle. A graphic of the FHWA “V” diagram of the Project Life Cycle is shown.  The main graphic is a V-shaped diagram with some additional horizontal “wings” on the left and right side of the top of the V.  Starting from the left “wing” the steps are Regional Architecture(s) and Feasibility Study / Concept Exploration.  At this point the steps begin to descend the left side of the V with: Concept of Operations; System Requirements; High-Level Design; and Detailed Design. The base of the V shows Software / Hardware Development & Field Installation.  At this point the steps begin to ascend the right side of the V with: Unit / Device Testing; Subsystem Verification; System Verification & Deployment; and System Validation.  Finally, the right “wing” includes steps that occur past the testing phases: Operations and Maintenance; Changes and Upgrades; and Retirement / Replacement. Each phase on the left of the “V” is associated by means of a dashed arrow with a phase on the right “V” (from the bottom): Detailed Design is associated with Unit / Device Testing by the Unit / Device Test Plan; High-Level Design is associated with Subsystem Verification by the Subsystem Verification Plan (Subsystem Acceptance); System Requirements is associated with System Verification & Deployment by the System Verification Plan (System Acceptance); and Concept of Operations is associated with System Validation by the System Validation Plan. The left side of the “V” is labeled “Decomposition and Definition”, the bottom is labeled “Implementation”, and the right side of the “V” is labeled “Integration and Recomposition”.  At the bottom, a rightward facing arrow is labeled “Time Line”.)

 

Slide 13:

Activity. A placeholder graphic with an image of hand over a computer keyboard to show that an activity is taking place.

 

Slide 14:

Learning Objective #1

Testing and Standardization

Why Perform Formal Testing?

Enter response in the chat pod

 

Slide 15:

Learning Objective #1

Review of Test Plans

A Test Plan is a Management-Level Document That Defines:

  1. What item is to be tested and when
  2. How it is to be tested
  3. Who is to test it
  4. In what detail the item is to be tested

Graphic of text. "Test Plan (TP) for NTCIP 1204 v03 ESS Standard."

 

Slide 16:

Learning Objective #1

Review of Test Plans (cont.)

What Item is to be Tested, and When is it to be Tested?

 

Slide 17:

Learning Objective #1

Review of Test Plans (cont.)

How is the Item to be Tested?

 

Slide 18:

Learning Objective #1

Review of Test Plans (cont.)

Who is to Perform the Testing?

 

Slide 19:

Learning Objective #1

Review of Test Plans (cont.)

In What Detail is the Item to be Tested?

 

Slide 20:

Test Plan Related Documents. Please see the Extended Text Description below.

(Extended Text Description: Test Plan Related Documents. In the title, “Test Plan-Related Documents”, Test Plan is underlined, and has an arrow leading to the first of three boxes, each of which respectively has an arrow to the next box, with the boxes containing “Test Design Specifications (TDS)” and text blurb "A document specifying the details of the test approach for a feature or combinatino of features and identifying the associated tests", “Test Case Specifications (TCS)” and text blurb "A document specifying inputs, predicted results, and a set of execution conditions for a test item" and “Test Procedure Specifications (TPS)” with text blurb "A document specifying a sequence of actions for the execution of a test.")

 

Slide 21:

Learning Objective #1

NTCIP 1204 and Test Documents

Which Documents are Included in the ESS Standard?

 

Slide 22:

Learning Objective #1

Test Plan Layout

Test Plan Layout. Please see the Extended Text Description below.

(Extended Text Description: Test Plan Layout. To the left of the text is a graphic showing the relationship of Test Plans and Test Specifications.  At the top is a block labeled “Test Plan”, which has a single arrow downward to a block labeled “Test Design Specification”.  This block has three arrows pointing to three blocks labeled “Test Case 1”, “Test Case 2”, and “Test Case 3”.  Below these three blocks are three blocks labeled “Test Procedure 1”, “Test Procedure 2”, and “Test Procedure 3”.  Test Cases 1 and 2 have arrows pointing to Test Procedure 1.  Test Case 3 has two arrows: the first points to Test Procedure 2; the second points to Test Procedure 3.)

 

Slide 23:

Learning Objective #2

NTCIP Family of Standards

 

Slide 24:

Learning Objective #2

NTCIP 1204 - ESS

 

Slide 25:

Learning Objective #2

NTCIP 1204 ESS Published Versions

 

Slide 26:

Learning Objective #2

Why Use NTCIP 1204 v03?

 

Slide 27:

Learning Objective #2

Elements of NTCIP 1204 Useful for Testing

 

Slide 28:

Learning Objective #2

Example NTCIP 1204 Dialog

4.2.14 Retrieve Pavement Surface Condition

The standardized dialog for a management station to retrieve precipitation presence shall be as follows:

a) (Precondition) The management station shall be aware of which row of the table is to be retrieved.

b) The management station shall GET the following objects for the pavement sensor of interest:

1) essSurfaceStatus.x

2) essSurfaceTemperature.x

3) essPavementSensorError.x

c) The management station shall GET pavementSensorModellnformation.x.

NOTE—This is an NTCIP 1204 v02 object that may result in a noSuchName. This level of detail was not provided in NTCIP 1204 v01.

 

Slide 29:

Polling. A placeholder slide showing hands raised signifying polling activities.

 

Slide 30:

Learning Objective #2

NTCIP 1204

Which of the following are included in NTCIP 1204 version 2?

  1. Protocol Requirements List
  2. Requirements to Test Cases Traceability Matrix
  3. Test cases
  4. Test procedures
  5. All of the above

 

Slide 31:

Case Study. A placeholder graphic showing a Traffic Management Center indicating that a Case Study follows.

 

Slide 32:

Learning Objective #3

Example of ESS Installation

ESS location includes:

A photo showing an example ESS installation. Please see the Extended Text Description below.

(Extended Text Description: A photo showing an example ESS installation.  A roadway with a single car is shown on the left of the photo.  On the right is a short tower with a control cabinet and temperature/humidity sensor at the bottom.  Toward the top are a traffic sensor and camera.  At the very top is a wind speed and direction sensor.)

Photo: Intelligent Devices, Inc.

 

Slide 33:

Learning Objective #3, 4

Test Documents for ESS using NTCIP 1204

For this example ESS installation, we will:

 

Slide 34:

Learning Objective #3

Sample Test Plan for ESS

Refer to Page 7 of the Student Supplement

Sample Test Plan Includes:

 

Slide 35:

Learning Objective #3

Sample Test Plan for ESS (cont.)

Refer to Pages 7-9 of the Student Supplement

 

Slide 36:

Learning Objective #3

Sample Test Plan for ESS (cont.)

Refer to Pages 10-11 of the Student Supplement

 

Slide 37:

Learning Objective #3

Test Design Specification (TDS)

 

Slide 38:

Learning Objective #3

TDS: ESS Features to be Tested

  1. Functional requirements are selected by the agency using the PRL in the NTCIP 1204.
  2. Functional requirements can be mandatory or optional.

Example: Functional requirement "Retrieve ESS Characteristics" is mandatory.

User Need ID

User Need

FR ID

Functional Requirement

Conformance

Project Requirement

Additional Project Requirements

F 1.2

Generic Features

M

Yes

F.1.2.1

Retrieve the Device Identity

M

Yes

3.5.1.1.1

Retrieve ESS Characteristics

M

Yes

3.5.1.1.3

Configure ESS Manager

M

Yes

3.5.2.1.1

Retrieve Atmospheric Pressure Height

Pressure:M

Yes NA

(Additional author's notes for this slide: Slide 38: Below the text, an excerpt from the NTCIP 1204 Protocol Requirements List (PRL) shows a table with the following columns: User Need ID, User Need, FR ID, Functional Requirement, Conformance, Project Requirement and Additional Project Requirements. Entries for User Need ID F.1.2 and F.1.2.1 are shown. FR ID 3.5.1.1.1, Retrieve ESS Characteristics, is outlined in a green box. FR ID 3.5.2.1.1 Project Requirements have “NA” highlighted in a red box.)

 

Slide 39:

Learning Objective #3

TDS: Features to be Tested (cont.)

User Need ID

User Need

FR ID

Functional Requirement

Conformance

Project Requirement

Additional Project Requirements

2.5.2

Sensor Manager Features

0.1 (1_*)

Yes / No

2.5.2.1 (Weather)

Monitor Weather Conditions

0.2 (1_*)

Yes/ No/ NA

2.5.2.1.1 (Pressure)

Monitor Atmospheric Pressure

0.3 (1_*)

Yes/ No/ NA

3.5.2.3.2.1

Retrieve Atmospheric Pressure

M

Yes/ NA

3.6.1

Required Number of Atmospheric Pressure Sensors

M

Yes/ NA

The ESS shall support at least atmospheric pressure sensors.

             

F.1.2

Generic Features

M

Yes

F.1.2.1

Retrieve the Device Identity

M

Yes

3.5.1.1.1

Retrieve ESS Characteristics

M

Yes

3.5.1.1.3

Configure ESS Manager

M

Yes

35.2.1.1

Retrieve Atmospheric Pressure Height

Yes / NA


(Additional author's notes for this slide: Slide 39: Below the text, an excerpt from the NTCIP 1204 PRL shows two tables with the same columns as defined in Slide 38. In the first table, User Need IDs 2.5.2 and 2.5.2.1 have Project Requirement “Yes” highlighted in a red box. Below User Need ID 2.5.2.1.1, the predicate “Pressure” has been highlighted in a green box, and the Project Requirement entry “No” has been highlighted in a red box. FR ID 3.5.2.3.2.1 an d3.6.1 both have Project Requirement entry “NA” highlighted in red boxes. In the second table, FR ID 3.5.2.1.1 has Conformance “Pressure: M” highlighted in a green box, and Project Requirement “NA” highlighted in a red box.)

 

Slide 40:

Learning Objective #3

TDS: Features to be Tested (cont.)

User Need ID

User Need

FR ID

Functional Requirement

Conformance

Project Requirement

Additional Project Requirements

2.5.2

Sensor Manager Features

0.1 (1_*)

Yes/ No

2.5.2.1 (Weather)

Monitor Weather Conditions

0.2 (1..*)

Yes/ No/ NA

2.5.2.1.1 (Pressure)

Monitor Atmospheric Pressure

0.3 (1..*)

Yes/ No/ NA

3.5.2.3.2.1

Retrieve Atmospheric Pressure

M

Yes/ NA

3.6.1

Required Number of Atmospheric Pressure Sensors

m

Yes/ NA

The ESS shall support at least atmospheric pressure sensors.

2.5.2.1.2 (Wind)

Monitor Winds

0.3 (1..*)

Yes/ No/ NA

3.5.2.3.2.2

Retrieve Wind Data

m

Yes/ NA

3.6.2

Required Number of Wind Sensors

M

Yes/ NA

The ESS shall support at least 1 wind sensors.

(Additional author's notes for this slide: Slide 40: Below the text, an excerpt from the NTCIP 1204 PRL shows a table with the same columns as defined in Slide 38. User Need IDs 2.5.2, 2.5.2.1 and 2.5.1.2 all have Project Requirements “Yes” highlighted in red boxes. User Need ID 2.5.2.1.1 has Project Requirement “No” highlighted in a red box. FR IDs 3.5.2.3.2.1 and 3.6.1 have Project Requirements “NA” highlighted in red boxes.)

 

Slide 41:

Learning Objective #3

TDS: Determine Test Case Using RTCTM

NTCIP 1204 v03 - Protocol Requirements List (PRL)

User Need ID

User Need

FR ID

Functional Requirement

Conformance

Project Requirement

Additional Project Requirements

2.5.2.1.2 (Wind)

Monitor Winds

0.3 (1..*)

Yes/ No / NA

3.5.2.3.2.2

Retrieve Wind Data

M

Yes/ NA

 

3.6.2

Required Number of Wind Sensors

M

Yes/ NA

The ESS shall support at least 1 wind sensors.

NTCIP 1204 v03/Requirements to Test Case Traceability Matrix (RTCTM)

Requirement

Test Case

ID

Title

ID

Title

 

C.2.3.3.2

Retrieve Atmospheric Pressure

3.5.2.3.2.2

Retrieve Wind Data

 

C.2.3.3.3\

Retrieve Wind Data

3.5.2.3.2.3

Retrieve Temperature

C.2.3.3.4

Retrieve Temperature

(Additional author's notes for this slide: Slide 41: Below the text, an excerpt from the NTCIP 1204 PRL shows a table with the same columns as defined in Slide 38. FR ID 3.5.2.3.2.2 is highlighted in a red box with an arrow leading to an excerpt from the NTCIP 1204 Requirements to Test Cases Traceability Matrix. The RTCTM has superheadings of Requirement and Test Cases, each with subheadings of ID and Title. Requirement ID 3.5.2.3.2.2 is highlighted in a red box, and pointed to by the arrow from the PRL table. Test Case ID C.2.3.3.3 is highlighted in a red box.)

 

Slide 42:

Learning Objective #3

Example Test Case: "Appropriate"

C.2.3.3.3 Retrieve Wind Data

Test Case: 3.3

Title:

Retrieve Wind Data

Description:

This test case verifies that the ESS allows a management station to determine current wind information.

Variables:

Required_ Wind_Sensors

PRL 3.6.2

Pass/Fail Criteria:

The device under test (DUT) shall pass every verification step included within the Test Case to pass the Test Case.

Step

Test Procedure

Device


5

FOR EACH value, N. from 1 to Supported Wind Sensors, perform Steps 5.1 through 5.22.

5.1

GET the following object{s): »windSensorAvg Speed. N swindSensorAvg Direction.N »windSensorSpotSpeed.N »windSensorSpotDirection N »windSensorGustSpeeci.N »windSensorGustDirection.N »windSensorSituation.N

Pass / Fail (Sec. 3.5.2.3.2.2)

5.2

VERIFY that the RESPONSE VALUE for wind Sen sorAvgSpeed.N is greater than or equal to 0.

Pass / Fail {Sec. 5.6.10 4)

5.3

VERIFY that the RESPONSE VALUE for windSensorAvgSpeed.N is less than or equal to 65535.

Pass / Fail {Sec 5.6.10.4)

5.4

VERIFY that the RESPONSE VALUE for WindSensorAvgSpeed.N is|APPROPRIATE |

Pass / Fail (Sec. 5.6.10.4)

(Additional author's notes for this slide: Slide 42: Below the heading, the title and heading box from NTCIP 1204 Test Case / Test Procedure C.2.3.3.3. Retrieve Wind Data. Below this, an excerpt from the Test Procedure, showing steps 5 through 5.4. In Step 5.4, the term “APPROPRIATE is outlined in a red box.)

 

Slide 43:

Polling. A placeholder slide showing hands raised signifying polling activities.

 

Slide 44:

Learning Objective #2

ESS Test Design - Approach

How is the Response Value Determined to be "Appropriate"?

Please see the Extended Text Description below.

(Extended Text Description: ESS Test Design - Approach. How is the Response Value Determined to be "Appropriate"? Below the subtitle, Step 5.4 from the Test Procedure is shown, "VERIFY that the RESPONSE VALUE for wind Sensor Avg Speed. N is APPROPRIATE" with the term “APPROPRIATE” outlined in a red box. Last section "Pass/Fail (Sec. 5.6.10.4).)

  1. Estimate of wind by on-site personnel
  2. By monitoring or simulating the output of the sensor -e.g., a variable voltage to simulate the output
  3. By simulating an input into the sensor - e.g., a motor used to rotate the propeller of the wind sensor
  4. By controlling the environment - .e.g., a calibrated wind tunnel
  5. All of the above

 

Slide 45:

Learning Objective #4

Test Case Specification (TCS)

 

Slide 46:

Learning Objective #4

Test Case Specifications (cont.)

IEEE 829 vs. NTCIP

C.2.3.3.3 Retrieve Wind Data

Test Case: 3.3

Title:

Retrieve Wind Data

Description:

This test case verifies that the ESS allows a management station to determine current wind information.

Variables:

Required_ Wind_Sensors

PRL 3.6.2

Pass/Fail Criteria:

The device under test (DUT) shall pass every verification step included within the Test Case to pass the Test Case.

Step

Test Procedure

Device

5

FOR EACH value, N, from 1 to Supported Wind Sensors, perform Steps 5.1 through 5.22.

(Additional author's notes for this slide: Slide 46: Below the text, an excerpt from the NTCIP 1204 Test Case / Test Procedure C.2.3.3.3 is shown, with “Test Case: 3.3” and “Test Procedure” highlighted in green boxes.)

 

Slide 47:

Learning Objective #4

Test Case Specifications (cont.)

Page 15 in the Supplement

NTCIP 1204 Test Case

Simulated Wind Speed

windSensorSpotSpeed.n

windSensorSituation.n

C.2.3.3.3

0 km/h

0

3
(calm)

C.2.3.3.3

> 118 km/h

> 118

11
(hurricaneForceWinds)

 

Slide 48:

Polling. A placeholder slide showing hands raised signifying polling activities.

 

Slide 49:

Learning Objective #4

Test Case Specifications

Which of the following are NOT defined in Test Case Specifications according to IEEE

  1. Inputs
  2. Execution conditions
  3. Steps to execute - these are defined in test procedure
  4. Expected results specifications
  5. All of the above

 

Slide 50:

Learning Objective #4

Test Procedures Specifications (TPS)

 

Slide 51:

Learning Objective #4

Test Procedures

Pages 16-17 in the Supplement

C.2.3.3.3 Retrieve Wind Data

Test Case: 3.3

Title:

Retrieve Wind Data

Description:

This test case verifies that the ESS allows a management station to determine current wind information.

Variables:

Required_ Wind_Sensors

PRL 3A2

Pass/Fail Criteria:

The device under test (DUT) shall pass every verification step included within the Test Case to pass the Test Case.

Step

Test Procedure

Device

1

CONFIGURE: Determine the number of wind sensors required by the specification {PRL 3 6 2) RECORD this information as: »Required_Wind_Sensors

2

GET the following object(s): swindSensorTableNnmSensors.O

Pass / Fail (Sec. 3.5.2.3.2.2)

 

Slide 52:

Learning Objective #4

Additional Test Documents

 

Slide 53:

What Have We Learned Today?

We Now Know How to Apply a Test Plan to the ESS Standard

 

Slide 54:

What Have We Learned Today? (cont.)

Describe :

 

Slide 55:

What Have We Learned Today? (cont.)

Identify and Use Key Elements of the NTCIP 1204 v03 Standard to Standardize Testing:

 

Slide 56:

Where to Find More Information

 

Slide 57:

Questions? A placeholder graphic image with word Questions? at the top, and an image of a lit light bulb on the lower right side.