Module 55 - T308

T308: Acceptance Testing for Advanced Transportation Controller (ATC) Application Programming Interface (API) Software

HTML of the PowerPoint Presentation

(Note: This document has been converted from a PowerPoint presentation to 508-compliant HTML. The formatting has been adjusted for 508 compliance, but all the original text content is included, plus additional text descriptions for the images, photos and/or diagrams have been provided below.)

 

Slide 1:

Welcome - Graphic image of introductory slide. Please see the Extended Text Description below.

(Extended Text Description: Welcome - Graphic image of introductory slide. A large dark blue rectangle with a wide, light grid pattern at the top half and bands of dark and lighter blue bands below. There is a white square ITS logo box with words "Standards ITS Training - Transit" in green and blue on the middle left side. The word "Welcome" in white is to the right of the logo. Under the logo box is the logo for the U.S. Department of Transportation, Office of the Assistant Secretary for Research and Technology.)

 

Slide 2:

Welcome slide with Ken Leonard and screen capture of home webpage. Please see the Extended Text Description below.

(Extended Text Description: This slide, entitled "Welcome" has a photo of Ken Leonard, Director, ITS Joint Program Office, on the left hand side, with his email address, Ken.Leonard@dot.gov. A screen capture snapshot of the home webpage is found on the right hand side - for illustration only - from August 2014. Below this image is a link to the current website: www.pcb.its.dot.gov - this screen capture snapshot shows an example from the Office of the Assistant Secretary for Research and Development - Intelligent Transportation Systems Joint Program Office - ITS Professional Capacity Building Program/Advanced ITS Education. Below the main site banner, it shows the main navigation menu with the following items: About, ITS Training, Knowledge Exchange, Technology Transfer, ITS in Academics, and Media Library. Below the main navigation menu, the page shows various content of the website, including a graphic image of professionals seated in a room during a training program. A text overlay has the text Welcome to ITS Professional Capacity Building. Additional content on the page includes a box entitled What's New and a section labeled Free Training. Again, this image serves for illustration only. The current website link is: http://www.pcb.its.dot.gov.)

 

Slide 3:

T308:

Acceptance Testing for Advanced Transportation Controller (ATC) Application Programming Interface (API) Software

This slide contains a graphic in a rounded rectangle that takes up about one-third the area of the slide and it located in the center lower two-thirds of the slide. Please see the Extended Text Description below.

(Extended Text Description: This slide contains a graphic in a rounded rectangle that takes up about one-third the area of the slide and it located in the center lower two-thirds of the slide. In the background of the rounded rectangle is a gray transportation controller. The controller is located in the upper right of the rounded rectangle and takes up about two-thirds of the rounded rectangle. The controller has two small keypads on the right side of the controller arranged vertically: the upper one has 4 rows by 3 columns of keys and the lower one has 4 rows by 4 columns of keys. There is a small screen in the center of the controller with the letters ATC over the top of it. There are three large circular connectors arranged horizontally below the screen. On the left-hand side of the controller there are two slots that appear to extend most of the height of the controller. The leftmost slot contains a collection of slender rectangular connectors. The slot to the right appears to be empty. In the foreground of the rounded rectangle is a left hand holding an orange USB flash drive. The hand appears in the lower left of the rounded rectangle and partially covers the view of the lower left of the transportation controller in the background. The USB flash drive has the letter "APIVS" on it.)

 

Slide 4:

Instructor

Headshot photo of Ralph W. Boaz

Ralph W. Boaz

President

Pillar Consulting, Inc.

San Diego, CA, USA

 

Slide 5:

Learning Objectives

  • Explain the purpose of the API Validation Suite (APIVS) Software
  • Use the API Reference Implementation (APIRI) test documentation to specify acceptance testing
  • Use the APIVS Software to test the API Software
  • Interpret and report results of testing API Software

 

Slide 6:

Learning Objective 1

  • Explain the purpose of the API Validation Suite (APIVS) Software

 

Slide 7:

API Software Testing in the Context of ATC Unit Testing

Quick Review of Advanced Transportation Controllers (ATCs)

  • A transportation controller is a computer
  • Traditional controllers run a single application program
  • Application Programming Interface (API) Software allows many application programs to run simultaneously
  • Application programs may come from different vendors than the ATC unit's manufacturer
  • Working, consistent and tested API Software is essential

 

Slide 8:

API Software Testing in the Context of ATC Unit Testing

Traditional controller units typically perform a single application

This slide lists traditional controller applications on the right and on the left is a graphic representing a transportation controller. Please see the Extended Text Description below.

(Extended Text Description: This slide lists traditional controller applications on the right and on the left is a graphic representing a transportation controller. The controller is about two and one-half times wider than it is tall. It has a rectangular screen center top and two key pads arranged horizontally underneath the screen. The keypad on the on the left has keys arranged in four rows by four columns. The keypad on the right has keys arranged in four rows by three columns. There is a small latch on the left-hand side of the controller. In the upper left-hand side of the controller are the words "MODEL 2070 CONTROLLER." There is a left bracket symbol between the controller on the left and the controller applications on the right. The bulleted text to the right reads:

Traditional Applications

  • Data Collection Application
    OR
  • Traffic Signal Application
    OR
  • Ramp Meter Application

)

Graphic: Ralph W. Boaz

 

Slide 9:

API Software Testing in the Context of ATC Unit Testing

ATC units can perform numerous applications simultaneously

This slide lists example ATC applications on the right and three graphics arranged vertically on the left. Please see the Extended Text Description below.

(Extended Text Description: This slide lists example ATC applications on the right and three graphics arranged vertically on the left. The top graphic is a rectangular blue box with the words in white "When using ATC API Software." Below the box are two transportation controllers. The upper controller is about twice as wide as it is tall. It has two small keypads on the right side of the controller arranged vertically: the upper one has 4 rows by 3 columns of keys and the lower one has 4 rows by 4 columns of keys. There is a small screen in the center of the controller with the letters "ATC" over the top of it. There are four large circular connectors arranged horizontally below the screen. On the left-hand side of the controller there are two slots that extend most of the height of the controller. The leftmost slot contains a collection of slender rectangular connectors. The slot to the right appears to be empty. The lower controller is about two and one-half times wider than it is tall. It has a rectangular screen left side top and two key pads arranged horizontally in the center top of the controller. The keypad on the on the left has keys arranged in four rows by four columns. The keypad on the right has keys arranged in four rows by three columns. Below the keypads there are several small rectangular connectors. In the upper right-hand side of the controller are the letters "ATC." There is a large left bracket symbol between the controller on the left and the controller applications on the right. The bulleted text on the right reads:

Example Applications for ATCs

  • Traffic Signal Control/Traffic Management
  • Transit/Light Rail Priority
  • Emergency Management
  • Lane Use
  • Red Light Enforcement
  • Speed Monitoring/Enforcement
  • Access Control
  • Advanced Traveler Information Systems (ATIS)
  • Data Collection Systems
  • Connected Vehicle (CV) Applications

)

Graphics: Ralph W. Boaz

 

Slide 10:

API Software Testing in the Context of ATC Unit Testing

Elements of API Software

  • Made up of three software libraries:
    • Front Panel User Interface (FPUI)
    • Field I/O (FIO)
    • Time of Day (TOD)
  • Two resource management programs:
    • Front Panel Manager
    • Field I/O Manager
  • Allows application developers to write programs that safely share the controller

This slide has a graphic of four interlocking gears of various sizes on the right side of the slide.

Graphic: Thinkstock

 

Slide 11:

API Software Testing in the Context of ATC Unit Testing

Example of the Front Panel Manager Window

This slide is a graphic illustrating how multiple concurrently running programs can operate on a single ATC controller unit. Please see the Extended Text Description below.

(Extended Text Description: This slide is a graphic illustrating how multiple concurrently running programs can operate on a single ATC controller unit. In the upper part of the slide is a square representing a screen and keypads of a transportation controller. Below the square is an enlargement of the screen. There are dotted lines extending from the screen in the controller to the edges of the enlargement. The enlarged screen is about 25% of the height of the slide and extends almost the entire width of the slide. The screen is 8 lines x 40 fixed width characters in size with the following text visible:

FRONT PANEL MANAGER VER 1.00

SELECT WINDOW: 0-F SET DEFAULT: *,0-F
0 Ramp Meter Prgrm 1*Signal Program
2 Emergency Mngmnt 3 Data Distributor
4 System Checker 5
6 7
8 9
[MORE- UP/DN ARROW] [CONFIG INFO- NEXT]

)

Graphic: Ralph W. Boaz

 

Slide 12:

API Software Testing in the Context of ATC Unit Testing

Unit Testing

Traditional controller unit testing:

  • Tests the controller hardware
    (may include the operating system)
  • Tests the application program running on the controller

n the upper right of the slide is a picture of State of California Transportation Lab's environmental chamber. Please see the Extended Text Description below.

(Extended Text Description: In the upper right of the slide is a picture of State of California Transportation Lab's environmental chamber. The view appears as looking through thick glass in the front of the chamber. There are two Model 2070 controllers arranged vertically in a metal rack. Each controller is about two and one-half times wider than it is tall. They have a rectangular screen center top and two key pads arranged horizontally underneath the screen. The keypad on the on the left has keys arranged in four rows by four columns. The keypad on the right has keys arranged in four rows by three columns. There is a small latch on the left-hand side of the controllers. There are cables attached to small connectors on the right-hand side middle of the front of the controllers. These cables run underneath the controllers to the back of the rack.)

The lower left side of the slide is a snapshot of an application program. Please see the Extended Text Description below.

(Extended Text Description: The lower left side of the slide is a snapshot of an application program. The program has about eighty green, yellow and red indicators. There are also about one hundred sixty small check boxes. The indicators and check boxes are arranged in various groupings of rows and columns.)

In the lower right of the slide is a picture of an open metallic suitcase. Please see the Extended Text Description below.

(Extended Text Description: In the lower right of the slide is a picture of an open metallic suitcase. The top part of the suit case is connected to the bottom part of the suit case by a chain along the left side of the suitcase which holds the top part in an open upward position. The inside of the upper portion has over one hundred thirty small green, yellow and red lights on a flat panel that fills the entire top portion of the suitcase. The lower part of the suitcase has approximately one hundred twenty toggle switches on a flat panel which takes up about two-thirds of the right side of the case. The left side of the lower portion of the suitcase has a space to store cables.)

Graphics: Econolite (U), Siemens (LL), McCain (LR)

 

Slide 13:

API Software Testing in the Context of ATC Unit Testing

API Validation Suite (APIVS) Software tests API Software

This slide contains several graphics. There are three blue rounded rectangles symbolic of transportation controllers. Please see the Extended Text Description below.

(Extended Text Description: This slide contains several graphics. There are three blue rounded rectangles symbolic of transportation controllers. They are all the same width with but they vary in height with the leftmost controller having about the same height and width, the middle controller having a 20% greater height than width, and the rightmost controller having 40% greater height than width. The leftmost controller has a rounded rectangle that is about three times the width than the height. It has four lines of dashes inside of it. It extends across the top of the controller and represents a controller screen. Below the screen are two rounded rectangles arranged horizontally both with four rows of four keys. These represent the controller keypads. Near the bottom of the controller are the letters "HW & O/S." The content of the middle controller is the same as the leftmost controller except that between the keypads and the text "HW &O/S" at the bottom is a rounded rectangle containing the text "App 1." The content of the rightmost controller is the same as the leftmost controller except that between the keypads and the text "HW &O/S" at the bottom are three rows of two rounded rectangles and a rectangle that extends the width of the controller. The rounded rectangles are labeled as follows: Row 1 – "App 1" and "App 2", Row 2 – "App 3" and "App 4", Row 3 – "App 5" and "App N." The rectangle below the rounded rectangle has the text "API SW." Beneath the left and middle controllers is a red text box with the text "APIVS Software tests API Software." There is a red rounded rectangle encircling the "API SW" rectangle in the rightmost controller. There is an arrow that extends from the red text box to the red rounded rectangle. The slide is animated. As the instructor discusses the types of testing, the controllers appear. When the instructor discusses testing the API software, the red text box, arrow and red rounded rectangle appear.)

Graphics: Ralph W. Boaz

 

Slide 14:

Architecture of the APIVS Software

Background

  • Four methods of software validation: inspection, demonstration, analysis, and test
  • Must validate that the API Software conforms to ATC 5401 Application Programming Interface Standard
  • API Validation Suite (APIVS) Software is used for testing
  • Testing involves initiating a test and comparing the result to a known correct result
  • Must be repeatable

 

Slide 15:

Architecture of the APIVS Software

Testing takes place inside the ATC unit

  • Computational power of the ATC unit allows internal testing of the API Software
  • Front panel and field I/O devices are emulated
  • "Loopback Drivers" cause the API Software's responses to be routed back to the APIVS Software
  • APIVS Software captures the test results in files and compares them to known correct results

 

Slide 16:

Architecture of the APIVS Software

Recall the layered ATC software architecture

This slide illustrates the layered architecture of ATC software. Please see the Extended Text Description below.

(Extended Text Description: This slide illustrates the layered architecture of ATC software. At the bottom of the graphic, centered on the slide, is a side view of a printed circuit board assembly referred to as the Engine Board. The Engine Board is about 60% the width of the slide. Above the Engine Board is a large yellow rectangle that is as wide as the Engine Board and twice the height. It is labeled "Linux Operating System and Device Drivers." Above the Linux Operating System and Device Drivers rectangle is a smaller red rectangle that is approximately one-third the width and one-half the height of the Linux Operating System and Device Drivers rectangle. It is labeled "API Software" and centered over the Linux Operating System and Device Drivers rectangle. Above the API Software rectangle is green rectangle of that is about 10% wider than the previous and the same height. It is labeled "Application Software." The Application Software rectangle is aligned to the right so that only 25% of the rectangle is over the API Software rectangle. Above the Application SW rectangle is a gray slightly smaller rounded rectangle. It is approximately one-third the width and the same height as the Application Software rectangle below it. It is labeled "Operational User" and it is centered above the API Software rectangle, the Linux Operating System and Device Drivers rectangle and the Engine Board. Between the Engine Board and the Linux Operating System and Device Drivers rectangle and between the Linux Operating System and Device Drivers rectangle and the API Software rectangle are heavy solid double arrows pointing upwards and downwards. Between the right side of the API Software rectangle and the left side of the Applications Software rectangle is a heavy solid double arrow pointing upwards and downwards. Between the right side of the API Software rectangle and the left side of the Application Software rectangle is a heavy solid double arrow pointing upwards and downwards. Between the right side of the Linux Operating System and Device Drivers rectangle and the right side of the Applications Software rectangle is a heavy solid double arrow pointing upwards and downwards. Centered between the API Software rectangle and the Operational User rounded rectangle is a heavy solid double arrow pointing upwards and downwards. There is a heavy solid double arrow pointing to the right side of the Operational User rounded rectangle and making a 90 degree turn downward to the top of the API Software rectangle. Between the Engine Board and the Linux Operating System and Device Drivers rectangle and between each of the rectangles above them are single heavy dashed lines running in parallel horizontally from a point just beyond the left side of the engine board to nearly the right side of the slide. These four lines delineate the five layers of the architecture. On the right side of the slide, arranged vertically, each layer of the architecture is identified top to bottom as "USERS," "APPLICATION LAYER," "API SOFTWARE LAYER," "ATC BOARD SUPPORT PACKAGE LAYER" and "HARDWARE LAYER." On the left side of the slide of is a broad blue double arrow pointing upwards and downwards that extends the complete height of the layered architecture.)

Graphic: Ralph W. Boaz

 

Slide 17:

Architecture of the APIVS Software

Modified architecture for the APIVS Software

This slide is very similar to the previous one except for the following changes. Please see the Extended Text Description below.

(Extended Text Description: This slide is very similar to the previous one except for the following changes:

  • In the gray rectangle, the word "Tester" replaces "Operational Use." There is now no 90 degree double arrow connecting the gray box and the API Software.
  • In the yellow rectangle, the words "API Validation Suite" replace "Application Software."
  • There is an additional green slender rectangle within the yellow Linux Operating System and Device Drivers rectangle. The new green rectangle is labeled "APIVS Loopback Drivers." There is a heavy solid double arrow pointing to the right side of the API Validation Suite rectangle and making a 90 degree turn downward to the top of the APIVS Loopback Drivers rectangle.
  • There is a large blue circle with broad lines in the middle of the slide overlaying the Linux Operating System and Device Drivers, APIVS Loopback Drivers, API Software, and API Validation Suite rectangles. This circle has two arrows along the line of the circle pointing in opposite directions. This circle replaces the broad blue line that was in the previous slide.

)

Graphics: Ralph W. Boaz

 

Slide 18:

Architecture of the APIVS Software

Detailed architecture of the APIVS Software

This slide shows a more detailed architecture of the APIVS software. Please see the Extended Text Description below.

(Extended Text Description: This slide shows a more detailed architecture of the APIVS software. There are two heavy dashed lines running in parallel horizontally across the slide delineating three layers of the APIVS architecture. On the right side of the slide, each layer of the architecture is identified top to bottom as "APPLICATION LAYER," "API SOFTWARE LAYER" and "ATC BOARD SUPPORT PACKAGE LAYER." In the ATC BOARD SUPPORT PACKAGE LAYER, there are two green ovals one on the left and right sides of the slide. The green ovals are labeled "VirtualLoopbackAsync" and "VirtualLoopbackSync," respectively. In the API SOFTWARE LAYER, aligned about green ovals are two red ovals labeled "Front Panel Manager (FPM)" and "Field I/O Manager (FIOM)," respectively. In centered between the two ovals is a red rounded rectangle labeled "API Libraries (FPUI, FIO, TOD)." In the APPLICATION LAYER, there is a green oval aligned over the API Libraries (FPUI, FIO, TOD) rounded rectangle. The green oval is labeled "Validation Suite Engine (VSE)." Above the Validation Suite Engine (VSE) are three pale yellow rectangular graphics that look like documents that are distributed across the top of the slide. They are labeled "Test Scripts (XML)," "Expected Result Files (Flat Files)," and "API Conformance Report (XML)," respectively. There are black arrows connecting the objects as follows:

  • There is an arrow extending from the Test Scripts (XML) object and pointing to the Validation Suite Engine (VSE) oval.
  • There is an arrow extending from the Expected Result Files (Flat Files) object and pointing to the Validation Suite Engine (VSE) oval.
  • There is an arrow extending from the Validation Suite Engine (VSE) oval and pointing to the API Conformance Report (XML) object.
  • There is a vertical double arrow connecting the Validation Suite Engine (VSE) oval and the API Libraries (FPUI, FIO, TOD) rounded rectangle.
  • There are horizontal arrows connecting the API Libraries (FPUI, FIO, TOD) rounded rectangle to the Front Panel Manager (FPM) oval and Field I/O Manager (FIOM) oval, respectively.
  • There is a vertical double arrow connecting the Front Panel Manager (FPM) oval and the Virtual-LoopbackAsync oval.
  • There is a vertical double arrow connecting the Field I/O Manager (FIOM) oval and the VirtualLoopbackSync oval.
  • There is a double arrow connecting the left side of the Virtual-LoopbackAsync oval upwards to the left side of the Validation Suite Engine (VSE) oval.
  • There is a double arrow connecting the right side of the VirtualLoopbackSync oval upwards to the right side of the Validation Suite Engine (VSE) oval.

A green rounded rectangle with not internal shading is used to highlight the three pale yellow documents at the top of the slide.)

Graphic: Ralph W. Boaz

 

Slide 19:

Features of the APIVS Software

Command-Line Interface (CLI) of the APIVS Software

This slide contains a graphic representing a Linux Command Shell window that has been accessed on the controller via a product called Tera Term. Please see the Extended Text Description below.

(Extended Text Description: This slide contains a graphic representing a Linux Command Shell window that has been accessed on the controller via a product called Tera Term. The window takes up almost the entire slide. It is all black except for a gray header that runs across the top 8% of the window. There are two lines of items in the gray header. The first line has a tiny computer icon with the words "COM6 – Tera Term VT". The second line has menu commands "File Edit Setup Control Window Help" with the first character of each command underlined. In the black portion of the window there is the following text in white characters:

# ATC login: root
# Password:
#
# vse -L 2 -c ./VS_config_1.txt -i C1420_in.xml –o C1420_log.xml

The line starting with "# vse" is encompassed by a yellow rounded rectangle.)

 

Slide 20:

Features of the APIVS Software

Command-Line Interface (CLI) of the APIVS Software

Command-Line Interface (CLI) of the APIVS Software. Please see the Extended Text Description below.

(Extended Text Description: Command-Line Interface (CLI) of the APIVS Software - in this slide, the first bulleted item is visible, the others are ghosted out:

vse-L [1-3][-c configuration-file] [-i APIVSXML-file] [-o output-file] [-n test_suite_name][-R count][-H][-C]

Where:

  • vse – Name of the VSE executable program.
  • -L [1-3]–(required) Level of output for the conformance report.
  • -c configuration-file –(optional) File that specifies a series of VSE configurable items. If this file is omitted, default values are used.
  • -i APIVSXML-file–(optional) Path to the input XML test script to use. If –i is not present, the input will be read from stdin.
  • -o output-file–(optional) Path of where to place the generated output XML file. If –o is not present, the output will be placed on stdout.

The "vse" in the command line and its definition are in dark green. The "-L [1-3]" in the command line and its definition are in blue. The "[-c configuration-file]" in the command line and its definition are in purple. The "[-i APIVSXML-file]" in the command line and its definition are in greenish-brown. The "[-o output-file]" in the command line and its definition are in light brown.

)

 

Slide 21:

Features of the APIVS Software

Command-Line Interface (CLI) of the APIVS Software

Command-Line Interface (CLI) of the APIVS Software. Please see the Extended Text Description below.

(Extended Text Description: Command-Line Interface (CLI) of the APIVS Software - in this slide, the second bulleted item is visible, the others are ghosted out:

vse-L [1-3][-c configuration-file] [-i APIVSXML-file] [-o output-file] [-n test_suite_name][-R count][-H][-C]

Where:

  • vse – Name of the VSE executable program.
  • -L [1-3]–(required) Level of output for the conformance report.
  • -c configuration-file –(optional) File that specifies a series of VSE configurable items. If this file is omitted, default values are used.
  • -i APIVSXML-file–(optional) Path to the input XML test script to use. If –i is not present, the input will be read from stdin.
  • -o output-file–(optional) Path of where to place the generated output XML file. If –o is not present, the output will be placed on stdout.

The "vse" in the command line and its definition are in dark green. The "-L [1-3]" in the command line and its definition are in blue. The "[-c configuration-file]" in the command line and its definition are in purple. The "[-i APIVSXML-file]" in the command line and its definition are in greenish-brown. The "[-o output-file]" in the command line and its definition are in light brown.

)

 

Slide 22:

Features of the APIVS Software

Command-Line Interface (CLI) of the APIVS Software

Command-Line Interface (CLI) of the APIVS Software. Please see the Extended Text Description below.

(Extended Text Description: Command-Line Interface (CLI) of the APIVS Software - in this slide, the third bulleted item is visible, the others are ghosted out:

vse-L [1-3][-c configuration-file] [-i APIVSXML-file] [-o output-file] [-n test_suite_name][-R count][-H][-C]

Where:

  • vse – Name of the VSE executable program.
  • -L [1-3]–(required) Level of output for the conformance report.
  • -c configuration-file –(optional) File that specifies a series of VSE configurable items. If this file is omitted, default values are used.
  • -i APIVSXML-file–(optional) Path to the input XML test script to use. If –i is not present, the input will be read from stdin.
  • -o output-file–(optional) Path of where to place the generated output XML file. If –o is not present, the output will be placed on stdout.

The "vse" in the command line and its definition are in dark green. The "-L [1-3]" in the command line and its definition are in blue. The "[-c configuration-file]" in the command line and its definition are in purple. The "[-i APIVSXML-file]" in the command line and its definition are in greenish-brown. The "[-o output-file]" in the command line and its definition are in light brown.

)

 

Slide 23:

Features of the APIVS Software

Command-Line Interface (CLI) of the APIVS Software

Command-Line Interface (CLI) of the APIVS Software. Please see the Extended Text Description below.

(Extended Text Description: Command-Line Interface (CLI) of the APIVS Software - in this slide, the fourth bulleted item is visible, the others are ghosted out:

vse-L [1-3][-c configuration-file] [-i APIVSXML-file] [-o output-file] [-n test_suite_name][-R count][-H][-C]

Where:

  • vse – Name of the VSE executable program.
  • -L [1-3]–(required) Level of output for the conformance report.
  • -c configuration-file –(optional) File that specifies a series of VSE configurable items. If this file is omitted, default values are used.
  • -i APIVSXML-file–(optional) Path to the input XML test script to use. If –i is not present, the input will be read from stdin.
  • -o output-file–(optional) Path of where to place the generated output XML file. If –o is not present, the output will be placed on stdout.

The "vse" in the command line and its definition are in dark green. The "-L [1-3]" in the command line and its definition are in blue. The "[-c configuration-file]" in the command line and its definition are in purple. The "[-i APIVSXML-file]" in the command line and its definition are in greenish-brown. The "[-o output-file]" in the command line and its definition are in light brown.

)

 

Slide 24:

Features of the APIVS Software

Command-Line Interface (CLI) of the APIVS Software

Command-Line Interface (CLI) of the APIVS Software. Please see the Extended Text Description below.

(Extended Text Description: Command-Line Interface (CLI) of the APIVS Software - in this slide, the last bulleted item is visible, the others are ghosted out:

vse-L [1-3][-c configuration-file] [-i APIVSXML-file] [-o output-file] [-n test_suite_name][-R count][-H][-C]

Where:

  • vse – Name of the VSE executable program.
  • -L [1-3] – (required) Level of output for the conformance report.
  • -c configuration-file –(optional) File that specifies a series of VSE configurable items. If this file is omitted, default values are used.
  • -i APIVSXML-file–(optional) Path to the input XML test script to use. If –i is not present, the input will be read from stdin.
  • -o output-file–(optional) Path of where to place the generated output XML file. If –o is not present, the output will be placed on stdout.

The "vse" in the command line and its definition are in dark green. The "-L [1-3]" in the command line and its definition are in blue. The "[-c configuration-file]" in the command line and its definition are in purple. The "[-i APIVSXML-file]" in the command line and its definition are in greenish-brown. The "[-o output-file]" in the command line and its definition are in light brown.

)

 

Slide 25:

Features of the APIVS Software

Command line interface for APIVS Software (cont.)

Command-Line Interface (CLI) of the APIVS Software. Please see the Extended Text Description below.

(Extended Text Description: Command line interface for APIVS Software (cont.) - in this slide, the first bulleted item is visible, the others are ghosted out:

vse-L [1-3][-c configuration-file] [-i APIVSXML-file] [-o output-file] [-n test_suite_name] [-R count] [-H][-C]

Where:

  • -n test_suite_name – (optional) Specific "test suite" named in the input XML file that is to be run. If omitted, all test suites contained in the file will be run.
  • -R count–(optional) Repeat test load count times, or indefinitely if count is 0.
  • -H–(optional) Halt on error when running in Repeat mode.
  • -C–(optional) Capture mode. Displays and command messages stored into files for use in subsequent tests.

The "-n test_suite_name" in the command line and its definition are in orange. The "[-R count]" in the command line and its definition are in dark blue. The "[-H]" in the command line and its definition are in light-green. The "[-C]" in the command line and its definition are in maroon.

)

 

Slide 26:

Features of the APIVS Software

Command line interface for APIVS Software (cont.)

Command-Line Interface (CLI) of the APIVS Software. Please see the Extended Text Description below.

(Extended Text Description: Command line interface for APIVS Software (cont.) - in this slide, the second bulleted item is visible, the others are ghosted out:

vse-L [1-3][-c configuration-file] [-i APIVSXML-file] [-o output-file] [-n test_suite_name] [-R count] [-H][-C]

Where:

  • -n test_suite_name–(optional) Specific "test suite" named in the input XML file that is to be run. If omitted, all test suites contained in the file will be run.
  • -R count–(optional) Repeat test load count times, or indefinitely if count is 0.
  • -H–(optional) Halt on error when running in Repeat mode.
  • -C–(optional) Capture mode. Displays and command messages stored into files for use in subsequent tests.

The "-n test_suite_name" in the command line and its definition are in orange. The "[-R count]" in the command line and its definition are in dark blue. The "[-H]" in the command line and its definition are in light-green. The "[-C]" in the command line and its definition are in maroon.

)

 

Slide 27:

Features of the APIVS Software

Command line interface for APIVS Software (cont.)

Command-Line Interface (CLI) of the APIVS Software. Please see the Extended Text Description below.

(Extended Text Description: Command line interface for APIVS Software (cont.) - in this slide, the third bulleted item is visible, the others are ghosted out:

vse-L [1-3][-c configuration-file] [-i APIVSXML-file] [-o output-file] [-n test_suite_name] [-R count] [-H][-C]

Where:

  • -n test_suite_name–(optional) Specific "test suite" named in the input XML file that is to be run. If omitted, all test suites contained in the file will be run.
  • -R count–(optional) Repeat test load count times, or indefinitely if count is 0.
  • -H–(optional) Halt on error when running in Repeat mode.
  • -C–(optional) Capture mode. Displays and command messages stored into files for use in subsequent tests.

The "-n test_suite_name" in the command line and its definition are in orange. The "[-R count]" in the command line and its definition are in dark blue. The "[-H]" in the command line and its definition are in light-green. The "[-C]" in the command line and its definition are in maroon.

)

 

Slide 28:

Features of the APIVS Software

Command line interface for APIVS Software (cont.)

Command-Line Interface (CLI) of the APIVS Software. Please see the Extended Text Description below.

(Extended Text Description: Command line interface for APIVS Software (cont.) - in this slide, the last bulleted item is visible, the others are ghosted out:

vse-L [1-3][-c configuration-file] [-i APIVSXML-file] [-o output-file] [-n test_suite_name] [-R count] [-H][-C]

Where:

  • -n test_suite_name–(optional) Specific "test suite" named in the input XML file that is to be run. If omitted, all test suites contained in the file will be run.
  • -R count–(optional) Repeat test load count times, or indefinitely if count is 0.
  • -H–(optional) Halt on error when running in Repeat mode.
  • -C–(optional) Capture mode. Displays and command messages stored into files for use in subsequent tests.

The "-n test_suite_name" in the command line and its definition are in orange. The "[-R count]" in the command line and its definition are in dark blue. The "[-H]" in the command line and its definition are in light-green. The "[-C]" in the command line and its definition are in maroon.

)

 

Slide 29:

Activity. A placeholder graphic with an image of hand over a computer keyboard to show that an activity is taking place.

 

Slide 30:

Question

What type of controller software is NOT traditionally tested by agencies?

Answer Choices

  1. Data Collection Software
  2. Signal Control Software
  3. Application Programming Interface Software
  4. Ramp Meter Software

 

Slide 31:

Review of Answers

A small graphical red and yellow X representing incorrect.a) Data Collection Software
Incorrect. Data Collection is an application. Agencies usually have methods for testing their applications.

A small graphical red and yellow X representing incorrect.b) Signal Control Software
Incorrect. Signal Control is an application. Agencies usually have methods for testing their applications.

A small graphical green and yellow check mark representing correct.c) Application Programming Interface Software
Correct! Until recently, it was not possible to test API software. The API Validation Suite discussed in this module provides this ability.

A small graphical red and yellow X representing incorrect.d) Ramp Meter Software
Incorrect. Ramp Meter is an application. Agencies usually have methods for testing their applications.

 

Slide 32:

Learning Objectives

  • Explain the purpose of the API Validation Suite (APIVS) Software
  • Use the API Reference Implementation (APIRI) test documentation to specify acceptance testing

 

Slide 33:

Learning Objective 2

  • Use the API Reference Implementation (APIRI) test documentation to specify acceptance testing

 

Slide 34:

API Reference Implementation (APIRI) Project

Illustrates the concept of how the transportation industry can maintain the reference implementation of the APIRI software. Please see the Extended Text Description below.

(Extended Text Description: This slide, entitled "Open Source API Reference Implementation," illustrates the concept of how the transportation industry can maintain the reference implementation of the APIRI software. In the center of the slide is picture of a computer tower. It is about 15% of the height of the slide and 5% of the width of the slide. It is a cuboid shape. There are buttons in the upper right of the tower, two horizontal slots for DVDs towards the top, a smaller slot for a 3.5" floppy disk and slats running down the lower 2/3 of the tower. Above the tower is the label "Open Source APIRI." There are four pictures arranged in an arch pattern over the tower. Each picture is approximately 15% of the height of the slide and 15% of the width of the slide. There are thick lined arrows pointing from each of the pictures to the computer tower.

  • On the left side of slide and aligned with the computer tower is a picture of a female electronics technician sitting at an electronics workbench with various tools. The technician is testing the components on a printed circuit board. Beneath the picture is the label "Manufacturers."
  • Not as far left as the Manufacturers picture and slightly above the level of the computer tower is a picture of a male computer professional working at a laptop computer. He is wearing a telephone earpiece. To the left of the picture is the label "Software Vendors."
  • On the right side of slide and aligned with the computer tower is a picture two male transportation professionals sitting in front of computer workstations. All or parts of six flat screen computer monitors are directly in front of them. In the background of the picture two walls can be seen. They are completely covered in video screens and monitors (approximately 50 in total). Beneath the picture is the label "Agencies."
  • Not as far right as the Agencies picture and slightly above the level of the computer tower is a picture of two males wearing neckties in front of two workstations with flat screen computer monitors. One of the males is pointing to the screen of the other. To the right of the picture is the label "Consultants."

Below the picture of the computer tower, is a large thick lined bracket. It is turned so that the open part of the bracket faces downward and extends almost the entire width of the slide. Below the bracket are pictures of four ATC controller units which are evenly distributed along the lower portion of the slide. The pictures are approximately 10% of the slide height and 15%-20% of the slide width. They are described below starting from the left.

  • The first controller is angled to the right. The unit is 10.75" W X 7.75" D X 10.5" H. On the lower one-third of the controller are four circular connectors which are evenly spaced across the width of the controller. They are labeled from left to right "A," "B," "C" and "D." To the left of the A connector are two small knobs. Above the circular connectors, on the left one-eighth of the controller are electronic cable connectors rotated 90 degrees so they are shown taller than wider. These include four RJ45 jacks, two 25 pin Sub D connectors, a 9 pin Sub D connector and a 15 pin Sub D connector. There are numerous small LEDs adjacent to the RJ45 jacks. Centered in the upper one-third area of the controller is an LCD screen. Below the screen are additional electronic connectors including four USB ports, six RJ45 jacks. To the right of the LCD screen is a keypad with 28 keys. Below the keypad and above the D connector is a knob. Facing the left side of the controller are two panels (1.5" W X 7" H each) which are held in place by two round thumbscrews (each).
  • The second controller is facing forward. The unit is 10" W X 10.5" D X 7" H. There is an LCD screen on the upper left portion of the controller. It is about 30% of the height and 40% of the width of the controller. To the right of the LCD screen is a keypad with 28 keys four of which appear to be arrows. Below the LCD screen is a knob and four RJ45 jacks. Below the knob is a circular connector. To the left of the circular connector are a knob and a small rectangular toggle switch. Beneath the RJ45 jacks are a 15 pin Sub D connector and three 9 pin Sub D connectors. To the right of the 9 pin Sub D connector and aligned with the right edge of the key pad are four USB ports. To the right of the keypad and USB ports is a panel (1.5" W X 7" H) which is held in place by two round thumbscrews.
  • The third controller is facing forward. The unit is 14.84" W X 6.13" D X 8.50" H. On the left one-eighth of the controller are electronic cable connectors rotated 90 degrees so they are shown taller than wider. These include two USB ports, four RJ45 jacks, a 25 pin Sub D connector, a 9 pin Sub D connector and a 15 pin Sub D connector. On the lower one-third of the controller and to the right of the electronic cable connectors are four circular connectors evenly spaced across the remaining width of the controller. They are labeled left to right "D," "C," "B" and "A." To the right of the A connector is a plastic screw about .25" in diameter. Above the D and C connectors is a LCD screen that takes up about 50% of the height and 40% of the width of the controller. To the right of the screen above the B and A connectors is a keypad with 24 keys. One of the keys is a large diamond shape which appears to have arrows on its corners.
  • The fourth controller is angled to the right. The unit is 19" W x 10.5" D x 7" H. There is an LCD screen on the upper left portion of the controller. It is about 25% of the width and 50% of the height of the controller. To the right of the LCD screen is a keypad with 32 keys. In the lower left front of the controller are two plastic screws about .25" in diameter. To the right of the screws below the left side of the LCD screen are two RJ45 jacks. To the right of the RJ45 jacks aligned with the right side of the LCD screen is a USB port. Below the key pad are a 15 pin Sub D connector, two 9 pin Sub D connectors, and a 25 pin Sub D connector. To the right of the 25 pin Sub D connector is a metal posted toggle switch. On the left and right sides of the controller are wire pull handles that are about 50% of the height of the controller aligned and centered vertically.

)

Graphics: Ralph W. Boaz (U)

Graphics: (L) Intelight, McCain, Econolite, Peek

 

Slide 35:

API Reference Implementation (APIRI) Project

  • USDOT funded the APIRI Project, which was completed in October 2016
  • Produced an open source software (OSS) implementation of ATC 5401 Standard v02 called the APIRI Software
  • Produced OSS APIVS Software to test API Software
  • Formal Verification and Validation process that can be used for testing any API Software implementation
  • APIRI Project Test documentation conforms to IEEE 829-2008

API RI Logo/icon

 

Slide 36:

API Reference Implementation (APIRI) Project

Benefits

  • Consistent with the Linux O/S open source concept
  • Promotes collaboration of developers across industry
  • Provides forum for users to express ideas and concerns
  • Promotes quick bug fixes and alternative solutions to issues
  • Facilitates introduction of new application developers
  • Incorporated on ATC units by manufacturers
  • Provides best opportunity for consistent API Software behavior

 

Slide 37:

Organization and Content of the APIRI Test Documentation

APIRI Documents Used to Specify Testing (IEEE Std 829-1998)

Organization and Content of the APIRI Test Documentation. Please see the Extended Text Description below.

(Extended Text Description: APIRI Documents Used to Specify Testing (IEEE Std 829-1998) - in this slide, the items in the purpose column of the table are ghosted out:

Test Document Purpose
Test Plan Specifies scope and approach for testing. Identifies the features to be tested by the Test Plan and, in the APIRI Project, includes the Test Design Specifications.
Test Design Specification (TDS) Specifies refinements of the test approach in the test plan and identifies the features to be tested by this design and the associated tests. There is a TDS in the Test Plan for each of the FPUI, FIO and TOD libraries.
Test Case Specification (TCS) Defines the information needed as it pertains to inputs and outputs from the software being tested. The APIRI project produced about 40 Test Case Specifications.
Test Procedure Specification (TPS) Specifies the steps for executing the test cases on the APIRI software. There are TPSs for testing using the APIVS software and TPSs for doing other methods of validation.

)

 

Slide 38:

Organization and Content of the APIRI Test Documentation

APIRI Documents Used to Specify Testing (IEEE Std 829-1998)

Organization and Content of the APIRI Test Documentation. Please see the Extended Text Description below.

(Extended Text Description: APIRI Documents Used to Specify Testing (IEEE Std 829-1998) - in this slide, the first item in the purpose column of the table is visible, the others are ghosted out:

Test Document Purpose
Test Plan Specifies scope and approach for testing. Identifies the features to be tested by the Test Plan and, in the APIRI Project, includes the Test Design Specifications.
Test Design Specification (TDS) Specifies refinements of the test approach in the test plan and identifies the features to be tested by this design and the associated tests. There is a TDS in the Test Plan for each of the FPUI, FIO and TOD libraries.
Test Case Specification (TCS) Defines the information needed as it pertains to inputs and outputs from the software being tested. The APIRI project produced about 40 Test Case Specifications.
Test Procedure Specification (TPS) Specifies the steps for executing the test cases on the APIRI software. There are TPSs for testing using the APIVS software and TPSs for doing other methods of validation.

)

 

Slide 39:

Organization and Content of the APIRI Test Documentation

APIRI Documents Used to Specify Testing (IEEE Std 829-1998)

Organization and Content of the APIRI Test Documentation. Please see the Extended Text Description below.

(Extended Text Description: APIRI Documents Used to Specify Testing (IEEE Std 829-1998) - in this slide, the second item in the purpose column of the table is visible, the others are ghosted out:

Test Document Purpose
Test Plan Specifies scope and approach for testing. Identifies the features to be tested by the Test Plan and, in the APIRI Project, includes the Test Design Specifications.
Test Design Specification (TDS) Specifies refinements of the test approach in the test plan and identifies the features to be tested by this design and the associated tests. There is a TDS in the Test Plan for each of the FPUI, FIO and TOD libraries.
Test Case Specification (TCS) Defines the information needed as it pertains to inputs and outputs from the software being tested. The APIRI project produced about 40 Test Case Specifications.
Test Procedure Specification (TPS) Specifies the steps for executing the test cases on the APIRI software. There are TPSs for testing using the APIVS software and TPSs for doing other methods of validation.

)

 

Slide 40:

Organization and Content of the APIRI Test Documentation

APIRI Documents Used to Specify Testing (IEEE Std 829-1998)

Organization and Content of the APIRI Test Documentation. Please see the Extended Text Description below.

(Extended Text Description: APIRI Documents Used to Specify Testing (IEEE Std 829-1998) - in this slide, the third item in the purpose column of the table is visible, the others are ghosted out:

Test Document Purpose
Test Plan Specifies scope and approach for testing. Identifies the features to be tested by the Test Plan and, in the APIRI Project, includes the Test Design Specifications.
Test Design Specification (TDS) Specifies refinements of the test approach in the test plan and identifies the features to be tested by this design and the associated tests. There is a TDS in the Test Plan for each of the FPUI, FIO and TOD libraries.
Test Case Specification (TCS) Defines the information needed as it pertains to inputs and outputs from the software being tested. The APIRI project produced about 40 Test Case Specifications.
Test Procedure Specification (TPS) Specifies the steps for executing the test cases on the APIRI software. There are TPSs for testing using the APIVS software and TPSs for doing other methods of validation.

)

 

Slide 41:

Organization and Content of the APIRI Test Documentation

APIRI Documents Used to Specify Testing (IEEE Std 829-1998)

Organization and Content of the APIRI Test Documentation. Please see the Extended Text Description below.

(Extended Text Description: APIRI Documents Used to Specify Testing (IEEE Std 829-1998) - in this slide, the last item in the purpose column of the table is visible, the others are ghosted out:

Test Document Purpose
Test Plan Specifies scope and approach for testing. Identifies the features to be tested by the Test Plan and, in the APIRI Project, includes the Test Design Specifications.
Test Design Specification (TDS) Specifies refinements of the test approach in the test plan and identifies the features to be tested by this design and the associated tests. There is a TDS in the Test Plan for each of the FPUI, FIO and TOD libraries.
Test Case Specification (TCS) Defines the information needed as it pertains to inputs and outputs from the software being tested. The APIRI project produced about 40 Test Case Specifications.
Test Procedure Specification (TPS) Specifies the steps for executing the test cases on the APIRI software. There are TPSs for testing using the APIVS software and TPSs for doing other methods of validation.

)

 

Slide 42:

Organization and Content of the APIRI Test Documentation

APIRI Documents Used to Specify Testing (IEEE Std 829-2008)

This slide contains a graphic illustrating the relationships of the test documents that were used to test the APIRI Software. Please see the Extended Text Description below.

(Extended Text Description: This slide contains a graphic illustrating the relationships of the test documents that were used to test the APIRI Software. Each document is represented by a graphic that has a rectangular-like shape except the bottom of the shape is curved. Each graphic has the name of the test document in the middle of it. When multiple documents of the same time are represented, the graphic is the same except that the right edges of two more documents are shown behind it. The graphic is arranged in four vertical areas or levels on the slide. At first level (top of slide) center of the slide is the document "APIRI Test Plan." In the second level, on the left side of the slide is the document "FPUI Test Design." On the right side of the slide is a rectangle with a dashed line labeled "FIO Test Design, etc." To the right of that, is another rectangle with a dashed line labeled "TOD Test Design, etc." There is a thin solid arrow extending from the APIRI Test Plan document to the FPUI Test Design document. There are thin dashed arrows extending from the APIRI Test Plan document to the dashed rectangles. In the third level, spaced evenly horizontally below the FPUI Test Design document, are two multiple documents that are labeled "FPUI Test Cases" and "FPUI Test Procedures," respectively. There are thin solid arrows extending from the FPUI Test Design document to the FPUI Test Cases and FPUI Test Procedures documents. There is a thin dashed double arrow extending horizontally between the FPUI Test Cases document and FPUI Test Procedures document. In the fourth level, there is a rectangle about 10 times as wide as it is tall made with solid lines and extending most of the page. The rectangle is labeled "Test Execution." There are solid arrows extending from the FPUI Test Cases and FPUI Test Procedures documents to the Test Execution rectangle.)

Graphic: Ralph W. Boaz

 

Slide 43:

Organization and Content of the APIRI Test Documentation

APIRI Documents Used to Specify Testing (IEEE Std 829-2008)

This slide contains a graphic illustrating the relationships of the test documents that were used to test the APIRI Software. Please see the Extended Text Description below.

(Extended Text Description: This slide contains a graphic illustrating the relationships of the test documents that were used to test the APIRI Software. Each document is represented by a graphic that has a rectangular-like shape except the bottom of the shape is curved. Each graphic has the name of the test document in the middle of it. When multiple documents of the same time are represented, the graphic is the same except that the right edges of two more documents are shown behind it. The graphic is arranged in four vertical areas or levels on the slide. At first level (top of slide) center of the slide is the document "APIRI Test Plan." In the second level, in the center of the slide is the document "FIO Test Design." On the left and right side of the slide are rectangles with dashed lines labeled "FPUI Test Design, etc." and "TOD Test Design, etc.," respectively. There is a thin solid arrow extending downward from the APIRI Test Plan document to the FIO Test Design document. There are thins dashed arrows extending from the APIRI Test Plan document to the dashed rectangles. In the third level, spaced evenly horizontally below the FIO Test Design document, are two multiple documents that are labeled "FIO Test Cases" and "FIO Test Procedures," respectively. There are thin solid arrows extending from the FIO Test Design document to the FIO Test Cases and FIO Test Procedures documents. There is a thin dashed double arrow extending horizontally between the FIO Test Cases document and FIO Test Procedures document. In the fourth level, there is a rectangle about 10 times as wide as it is tall made with solid lines and extending most of the page. The rectangle is labeled "Test Execution." There are solid arrows extending from the FIO Test Cases and FIO Test Procedures documents to the Test Execution rectangle.)

Graphic: Ralph W. Boaz

 

Slide 44:

Organization and Content of the APIRI Test Documentation

APIRI Documents Used to Specify Testing (IEEE Std 829-2008)

This slide contains a graphic illustrating the relationships of the test documents that were used to test the APIRI Software. Please see the Extended Text Description below.

(Extended Text Description: This slide contains a graphic illustrating the relationships of the test documents that were used to test the APIRI Software. Each document is represented by a graphic that has a rectangular-like shape except the bottom of the shape is curved. Each graphic has the name of the test document in the middle of it. When multiple documents of the same time are represented, the graphic is the same except that the right edges of two more documents are shown behind it. The graphic is arranged in four vertical areas or levels on the slide. At first level (top of slide) center of the slide is the document "APIRI Test Plan." In the second level, or the right side of slide is the document "TOD Test Design." On the left side and center of the slide are rectangles with dashed lines labeled "FPUI Test Design, etc." and "FIO Test Design, etc.," respectively. There is a thin solid arrow extending downward from the APIRI Test Plan document to the TOD Test Design document. There are thins dashed arrows extending from the APIRI Test Plan document to the dashed rectangles. In the third level, spaced evenly horizontally below the TOD Test Design document, are two multiple documents that are labeled "TOD Test Cases" and "TOD Test Procedures," respectively. There are thin solid arrows extending from the TOD Test Design document to the TOD Test Cases and TOD Test Procedures documents. There is a thin dashed double arrow extending horizontally between the TOD Test Cases document and TOD Test Procedures document. In the fourth level, there is a rectangle about 10 times as wide as it is tall made with solid lines and extending most of the page. The rectangle is labeled "Test Execution." There are solid arrows extending from the TOD Test Cases and TOD Test Procedures documents to the Test Execution rectangle.)

Graphic: Ralph W. Boaz

 

Slide 45:

Organization and Content of the APIRI Test Documentation

APIRI Test Plan Outline. Please see the Extended Text Description below.

(Extended Text Description: In discussing Section 3 Features to Be Tested, the section number and title are encompassed by a rounded rectangle. The full text is listed below:

APIRI Test Plan Outline

1 - Introduction

2 - Test Items

3 - Features to Be Tested

4 - Features Not to Be Tested

5 - Approach

6 - Item Pass/Fail Criteria

7 - Suspension Criteria and Resumption Requirements

8 - Test Deliverables

9 - Testing Tasks

10 - Environmental Needs)

 

Slide 46:

Organization and Content of the APIRI Test Documentation

Features to Be Tested

Features to Be Tested. Please see the Extended Text Description below.

(Extended Text Description: As each column in the table is discussed, the column is encompassed by a rounded rectangle. As the next column is discussed, the previous rounded rectangle disappears. After the columns are discussed, the row "APIRI.TCS.2040 APIRI Test Case Spec 4 FPUI Reading and Writing Data" is encompassed by a rounded rectangle. The full table data is below:

Test ID Document Name Brief Description
APIRI.TDS.2001 APIRI Test Design Spec 1 Test All APIRI FPUI Required Features
APIRI.TDS.3001 APIRI Test Design Spec 2 Test All APIRI FIO Required Features
APIRI.TDS.4001 APIRI Test Design Spec 3 Test All APIRI TOD Required Features
APIRI.TCS.2010 APIRI Test Case Spec 1 FPUI Text UI Virtual Displays
APIRI.TCS.2020 APIRI Test Case Spec 2 FPUI Front Panel Manager
APIRI.TCS.2030 APIRI Test Case Spec 3 FPUI Character Set and Screen Attribs
APIRI.TCS.2040 APIRI Test Case Spec 4 FPUI Reading and Writing Data
APIRI.TCS.2100 APIRI Test Case Spec 9 API Version Information (All Libraries)
APIRI.TCS.3010 APIRI Test Case Spec 10 General FIO Operations
APIRI.TCS.3020 APIRI Test Case Spec 11 FIO Inputs and Outputs
APIRI.TCS.3030 APIRI Test Case Spec 12 FIO Channel Mapping
...    

)

 

Slide 47:

Organization and Content of the APIRI Test Documentation

APIRI Test Plan Outline (cont.). Please see the Extended Text Description below.

(Extended Text Description: In discussing Sections 16.1, 16.2, 16.3 and 16.4, the section numbers and titles are encompassed by a rounded rectangle. The full text is listed below:

APIRI Test Plan Outline (cont.)

11 - Responsibilities

12 - Staffing and Training Needs

13 - Schedule

14 - Risks and Contingencies

15 - Approvals

16 - Appendices

16.1 - FPUI Library Requirements to Validation Description Matrix

16.2 - FIO Library Requirements to Validation Description Matrix

16.3 - TOD Library Requirements to Validation Description Matrix

16.4 - APIRI Test Design Specifications

)

 

Slide 48:

Organization and Content of the APIRI Test Documentation

FPUI Library Requirements to Validation Description Matrix

Organization and Content of the APIRI Test Documentation. Please see the Extended Text Description below.

(Extended Text Description: As each column in the table is discussed, the column is encompassed by a rounded rectangle. As the next column is discussed, the previous rounded rectangle disappears. After the columns are discussed, the row beginning with "APIR3.1.1.2[10]" is encompassed by a rounded rectangle. The full table data is below:

Req ID Req Description ATC API Function APIRI SDD Design Narrative Test Cases Test Procedures
... ... ... ... ... ...
APIR3.1. 1.2[10] The API shall provide a function to read a queued character or key code from the input buffer of a window. fpui_read_ char The implementation of the fpui_read_char() library function (Section 3.4.8) makes use of the Linux operating system call to return a single character from the input buffer of the FrontPanelDriver device interface (Section 3.3). APIRI.TCS. 2040 APIRI.TPS. 1001
APIR3.1. 1.2[11] The API shall provide a function to write a character to the current cursor position of a window. fpui_write _char The implementation of the fpui_write_char() library function (Section 3.4.8) makes use of the Linux operating system call to write a single character to the output buffer of the FrontPanelDriver device interface (Section 3.3). APIRI.TCS. 2040 APIRI.TPS. 1001
... ... ... ... ... ...

)

 

Slide 49:

Organization and Content of the APIRI Test Documentation

APIRI Test Design Outlines

16.4.1 - Test Design Specification 1 - Test All APIRI FPUI Features

16.4.1.1 - Test Design Specification Identifier

The identifier for this Test Design Specification is APIRI.TDS.2001.

16.4.1.2 - Features To Be Tested

This Test Design Specification will test all FPUI features of the API Reference Implementation (APIRI) which are subject to testing for validation…

16.4.1.3 - Approach Refinements

All test cases will be tested using the general approach as defined in this test plan and as further refined in Test Procedure Specification APIRI.TPS.0001…

 

Slide 50:

Organization and Content of the APIRI Test Documentation

APIRI Test Design Outlines (cont.)

16.4.1.4 - Test Identification

All test documents to be used by this Test Design Specification can be found in Section 3, Table 1.

16.4.1.5 - Feature Pass/Fail Criteria

This Test Design Specification will be considered to have passed if and only if every individual test case passes according to its own pass/fail criteria as well as any pass/fail criteria associated with the test procedure used to execute the test case.

 

Slide 51:

Organization and Content of the APIRI Test Documentation

APIRI Test Case Outlines

2.6 - Test Case Specification 4 - FPUI Reading and Writing Data

2.6.1 - Test Case Specification Identifier The identifier for this Test Case Specification is APIRI.TCS.2040.

2.6.2 - Objective

The objective of this Test Case is to test the operation of the API functions used to write display data to and read keypresses from the Front Panel.

2.6.3 - Test Items

...

APIR3.1.1.2[13] The API shall provide a function to write a string to a window at the current cursor position.

APIR3.1.1.2[14] The API shall provide a function to write a string to a window at a starting position defined by column number and line number.

 

Slide 52:

Organization and Content of the APIRI Test Documentation

APIRI Test Case Outlines (cont.)

2.6.4 - Input Specifications

This test case requires the following file(s) as input:

File Description
C2040_in.xml APIVSXML test script (XML format)
Cxxxx_key0.txt keystroke file (Key '0')
Cxxxx_key1.txt keystroke file (Key '1')
Cxxxx_keyESC.txt keystroke file (Key '<Esc>')
C2040_vd_1.txt Virtual Display compare file (display 1)
VS_config_1.txt VSE configuration file (for VSE command line)

2.6.5 - Output Specifications

This test case produces the following file(s) as output:

File Description
C2040_log.xml Conformance report (XML format)

 

Slide 53:

Organization and Content of the APIRI Test Documentation

APIRI Test Procedure Outlines

2.1 - Test Procedure Specification 1 - Auto-Execute Selected APIVS Script(s)

2.1.1 - Test Procedure Specification Identifier

The identifier for this Test Procedure Specification is APIVS.TPS.1001.

2.1.2 - Purpose

This procedure runs the Validation Suite Engine (VSE) using the source test script and runtime options as associated with one or more specific Test Case Specifications. This execution will run from beginning to end with only limited human intervention.

 

Slide 54:

Organization and Content of the APIRI Test Documentation

APIRI Test Procedure Outlines (cont.)

2.1.3 - Special Requirements

This procedure requires the editing of text files and the movement of files between a host computer Hard Disk Drive and a USB Flash Drive and is intended to be run by an operator with a reasonable technical knowledge of personal computer (PC) file systems...

2.1.4 - Procedure Steps

Subsections contained in this section: Log, Setup, Start, Proceed, Measure, Shutdown, Restart, Stop, Wrap Up, and Contingencies

 

Slide 55:

Files Needed for Executing the Test Cases

Files Needed for Executing the Test Cases. Please see the Extended Text Description below.

(Extended Text Description: Files Needed for Executing the Test Cases - in this slide, the items in the purpose column of the table are ghosted out:

Test Document Purpose
APIRI Test Scripts Written in XML (extensible markup language), which allows testers to exercise the API software without having to write C programs. These Test Scripts are input to the VSE.
Flat Files Files that are used to configure the device emulators in the APIVS software and files that represent known correct outputs of the API software for given test cases.
Validation Suite Engine (VSE) Configuration File Allows testers to set various system options for APIVS software such as the file paths, screen size, and setting the ports for the loopback device driver software.
Linux Shell Scripts Allows the testers to run successive executions of the VSE without typing them in a line at a time.
Output Files Output from the VSE in XML format, which allows various tools to be used for analyzing test results.

)

 

Slide 56:

Files Needed for Executing the Test Cases

Files Needed for Executing the Test Cases. Please see the Extended Text Description below.

(Extended Text Description: Files Needed for Executing the Test Cases - in this slide, the first item in the purpose column of the table is visible, the others are ghosted out:

Test Document Purpose
APIRI Test Scripts Written in XML (extensible markup language), which allows testers to exercise the API software without having to write C programs. These Test Scripts are input to the VSE.
Flat Files Files that are used to configure the device emulators in the APIVS software and files that represent known correct outputs of the API software for given test cases.
Validation Suite Engine (VSE) Configuration File Allows testers to set various system options for APIVS software such as the file paths, screen size, and setting the ports for the loopback device driver software.
Linux Shell Scripts Allows the testers to run successive executions of the VSE without typing them in a line at a time.
Output Files Output from the VSE in XML format, which allows various tools to be used for analyzing test results.

)

 

Slide 57:

Files Needed for Executing the Test Cases

Files Needed for Executing the Test Cases. Please see the Extended Text Description below.

(Extended Text Description: Files Needed for Executing the Test Cases - in this slide, the second item in the purpose column of the table is visible, the others are ghosted out:

Test Document Purpose
APIRI Test Scripts Written in XML (extensible markup language), which allows testers to exercise the API software without having to write C programs. These Test Scripts are input to the VSE.
Flat Files Files that are used to configure the device emulators in the APIVS software and files that represent known correct outputs of the API software for given test cases.
Validation Suite Engine (VSE) Configuration File Allows testers to set various system options for APIVS software such as the file paths, screen size, and setting the ports for the loopback device driver software.
Linux Shell Scripts Allows the testers to run successive executions of the VSE without typing them in a line at a time.
Output Files Output from the VSE in XML format, which allows various tools to be used for analyzing test results.

)

 

Slide 58:

Files Needed for Executing the Test Cases

Files Needed for Executing the Test Cases. Please see the Extended Text Description below.

(Extended Text Description: Files Needed for Executing the Test Cases - in this slide, the third item in the purpose column of the table is visible, the others are ghosted out:

Test Document Purpose
APIRI Test Scripts Written in XML (extensible markup language), which allows testers to exercise the API software without having to write C programs. These Test Scripts are input to the VSE.
Flat Files Files that are used to configure the device emulators in the APIVS software and files that represent known correct outputs of the API software for given test cases.
Validation Suite Engine (VSE) Configuration File Allows testers to set various system options for APIVS software such as the file paths, screen size, and setting the ports for the loopback device driver software.
Linux Shell Scripts Allows the testers to run successive executions of the VSE without typing them in a line at a time.
Output Files Output from the VSE in XML format, which allows various tools to be used for analyzing test results.

)

 

Slide 59:

Files Needed for Executing the Test Cases

Files Needed for Executing the Test Cases. Please see the Extended Text Description below.

(Extended Text Description: Files Needed for Executing the Test Cases - in this slide, the fourth item in the purpose column of the table is visible, the others are ghosted out:

Test Document Purpose
APIRI Test Scripts Written in XML (extensible markup language), which allows testers to exercise the API software without having to write C programs. These Test Scripts are input to the VSE.
Flat Files Files that are used to configure the device emulators in the APIVS software and files that represent known correct outputs of the API software for given test cases.
Validation Suite Engine (VSE) Configuration File Allows testers to set various system options for APIVS software such as the file paths, screen size, and setting the ports for the loopback device driver software.
Linux Shell Scripts Allows the testers to run successive executions of the VSE without typing them in a line at a time.
Output Files Output from the VSE in XML format, which allows various tools to be used for analyzing test results.

)

 

Slide 60:

Files Needed for Executing the Test Cases

Files Needed for Executing the Test Cases. Please see the Extended Text Description below.

(Extended Text Description: Files Needed for Executing the Test Cases - in this slide, the last item in the purpose column of the table is visible, the others are ghosted out:

Test Document Purpose
APIRI Test Scripts Written in XML (extensible markup language), which allows testers to exercise the API software without having to write C programs. These Test Scripts are input to the VSE.
Flat Files Files that are used to configure the device emulators in the APIVS software and files that represent known correct outputs of the API software for given test cases.
Validation Suite Engine (VSE) Configuration File Allows testers to set various system options for APIVS software such as the file paths, screen size, and setting the ports for the loopback device driver software.
Linux Shell Scripts Allows the testers to run successive executions of the VSE without typing them in a line at a time.
Output Files Output from the VSE in XML format, which allows various tools to be used for analyzing test results.

)

 

Slide 61:

Files Needed for Executing the Test Cases

APIRI Test Scripts in XML

ATC 5401 API Reference Implementation Project

Filename: C2 040_in.xml

File Type: APIVSXML test script (XML format)

Test Case: APIRI.TCS.2040

Description: FPUI Reading and Writing Data TC XML: begins on Line 1187

Test Case Narrative

open an FPUI connection

put the app in focus, wait for confirmation

write to the VD using all write methods

(APIR3.1.1.2[15])

(APIR3.1.1.2[16])

(APIR3.1.1.2[11])

...

 

Slide 62:

Files Needed for Executing the Test Cases

APIRI Test Scripts in XML (cont.)

APIRI Test Scripts in XML (cont.). Please see the Extended Text Description below.

(Extended Text Description: As the instructor discusses areas of the APIRI Test Script, they are encompassed by rounded rectangles. As the discussion moves to the next area of the APIRI Test Script, the previous rounded rectangle disappears. The items encompassed by rounded rectangles are as follows:

  • The line that begins with "<!—" and the 5 lines that begin with "<Set"
  • The line that begins "<Call ref="fpui_write""
  • The line that begins "<Call ref="fpui_write_at""
  • The line that begins "<Call ref="fpui_write_char""

The full text is below:

<!-- write to the VD using all write method -->
<Set var="$write_buf" value="@C2040"/>
<Set var="$write_chr" value="@J"/>
<Set var="$write_len" value="%1"/>
<Set var="$row" value="%4"/>
<Set var="$column" value="%6"/>
<!-- (APIR3.1.1.2[15]) -->
<Call ref="fpui_write" setUp="API_Init_Variables"/>
<!-- (APIR3.1.1.2[16]) -- >
<Call ref="fpui_write_at" setUp="API_Init_Variables"/>
<!-- (APIR3.1.1.2[11]) -- >
<Call ref="fpui_write_char" setUp="API_Init_Variables"/>
...

)

 

Slide 63:

Files Needed for Executing the Test Cases

Expected Result Flat Files

Expected Result Flat Files. Please see the Extended Text Description below.

(Extended Text Description: As the instructor discusses the expected result flat file, the lines of the file starting with the "FONT PANEL MANAGER" line through the "[UP/DN ARROW] [CONFIG INFO- NEXT]" line are encompassed by a rounded rectangle. The full text is below:

# -- Virtual Display and Global Variable Dump -

# Date: 20160713 19:55:26 -#

# Display Rows:

#0 1 2 3 4

#2345678901234567890123456789012345678901234567 89

FRONT PANEL MANAGER

SELECT WINDOW [0-F] SET DEFAULT *[0-F]
0 C1160_00 1 C1160_01
2 C1160_02 3 C1160_03
4 C1160_04 5 C1160_05
6 C1160_06 7 C1160_07
8 C1160_08 9 C1160_09
[UP/DN ARROW] [CONFIG INFO- NEXT]

#----------------------------------------

)

 

Slide 64:

Files Needed for Executing the Test Cases

Expected Result Flat Files (cont.)

Files Needed for Executing the Test Cases. Please see the Extended Text Description below.

(Extended Text Description: As the instructor discusses the expected result flat file, the lines of the file starting with "0x" are encompassed by a rounded rectangle. The full text is below:

# ATC 5401 API Reference Implementation Project
#
# Filename: C3 02 0_cmd55a.txt
# File Type: APIVS flat file (text format)
# Test Case: APIRI.TCS.3xxx
# Description: file load of Command Frame 55, test outputs set
#
# Date Revision Description
# 2/24/16 1.0 initial release
0x37 0x05 0x50 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
0x00 0x00 0x00 0x00 0x00

)

 

Slide 65:

Files Needed for Executing the Test Cases

Validation Suite Engine (VSE) Configuration File

# Filename: VS_config_1.txt
# File Type: VSE configuration file
# Test Case: many
# Description:
#
# Date Revision Description
# 10/21/15 1.0 initial release
XMLInputFilePath = ./
XMLOutputFilePath = /tmp/
SetFilePath = ./
ScreenHeight = 8
ScreenWidth = 40
FPUICompareFilePath = ./
FPUIInputFilePath = ./

 

Slide 66:

Activity. A placeholder graphic with an image of hand over a computer keyboard to show that an activity is taking place.

 

Slide 67:

Question

What document is used to specify the inputs and outputs for a particular test of the API Software?

Answer Choices

  1. Test Design Specification
  2. Test Procedure Specification
  3. Test Plan
  4. Test Case Specification

 

Slide 68:

Review of Answers

A small graphical red and yellow X representing incorrect.a) Test Design Specification
Incorrect. The TDS specifies refinements of the test approach in the test plan.

A small graphical red and yellow X representing incorrect.b) Test Procedure Specification
Incorrect. The TPS specifies the steps for executing the test cases on the API software.

A small graphical red and yellow X representing incorrect.c) Test Plan
Incorrect. The test plan specifies the scope and approach for testing and identifies the features to be tested. In the APIRI Project, it also contains the Test Design Specifications.

A small graphical green and yellow check mark representing correct.d) Test Case Specification
Correct! In the APIRI Project, all of the tests cases are contained in a single document.

 

Slide 69:

Learning Objectives

  • Explain the purpose of the API Validation Suite (APIVS) Software
  • Use the API Reference Implementation (APIRI) test documentation to specify acceptance testing
  • Use the APIVS Software to test the API Software

 

Slide 70:

Learning Objective 3

  • Use the APIVS Software to test the API Software

 

Slide 71:

Open Source Software (OSS) Environment of the APIVS Software

APIVS Repository - https://github.com/apiriadmin/APIVS

Open Source Software (OSS) Environment of the APIVS Software. Please see the Extended Text Description below.

(Extended Text Description: This slide has a screen shot of the APIVS repository. At the top of the screenshot on the left side is "apiriadmin / APIVS." On the top right side of the screen are three boxes. The first box has an eye symbol, the word "Watch" and a 6. The second box has a star symbol, the word "Star" and a 0. The third box has a fork symbol, the word "Fork," and a 5. The next row appears to be tabs that are selectable by the user. There are various small symbols next to each tab representative of their function. The tabs in order left to right are "Code," "Issues 2," "Pull requests 0," "Projects 0," "Pulse" and "Graphs." It appears that the Code tab is being displayed. Below the tables is a title "APIVS Repository." Below the title is a slender rectangle that extends across the screen. In it are the items on a single line as follows: "123 commits," "1 branch," "0 releases," "5 contributors" and "GPL-2.0." Each of the items has a small symbol next to the representative of their function. Below the rectangle there are four rectangular buttons, two on the left and two on the right of the screenshot. The leftmost button says "Branch: master." The button just to the right of it says "New pull request." The rightmost button says "Clone or download." The button just to the left of it says "Find file." The bottom one-half of the screen shot appears to be a table representative of a file system. The first line says "apiriadmin committed on GitHub Merge pull request #18 from gallagjm/master … Latest commit 3684dd6 on Aug 22, 2016." The remaining lines have information arranged in columns as follows:

docs Add project test documentation. 3 months ago
include Remove older copyright notices. 2 months ago
modules Add APIVS source code, a year ago
samples APIVS update with updated demo test file and package script a year ago
src Remove older copyright notices. 2 months ago
support Add full set of test cases. 2 months ago
tests Add run scripts and support files for miscellaneous non-automated tests. 2 months ago

)

 

Slide 72:

Preparations for Testing

Equipment Required for Testing

  • Basic Equipment
    • ATC unit with operational API Software
    • PC with 1GB available hard drive storage and USB port
    • VSE executable provided by your ATC vendor (or compiled by you using vendor's tool chain)
  • If using CLI Method add
    • Serial or Ethernet cable to connect the PC to the ATC unit
  • If using USB Test Package Method add
    • 1GB USB Flash Drive (minimum), formatted with a FAT16 or FAT32 file system

 

Slide 73:

Preparations for Testing

Command-Line Interface (CLI) Method

  • APIVS Software has CLI designed to run in a Linux "shell"
  • Allows complete control for each execution of a test
  • Best method if tester is doing a lot of variations on individual tests or creating new tests
  • Tester must be comfortable working in a Linux environment

This slide has a graphic that shows a configuration used for testing API software. Please see the Extended Text Description below.

(Extended Text Description: This slide has a graphic that shows a configuration used for testing API software. The graphic is in the lower one-half of the slide. There is a transportation controller positioned a little right of center of the slide. It is about twice as wide as it is tall. It has two small keypads on the right side of the controller arranged vertically: the upper one has 4 rows by 3 columns of keys and the lower one has 4 rows by 4 columns of keys. There is a small screen in the center of the controller with the letters "ATC" over the top of it. There are four large circular connectors arranged horizontally below the screen. On the left-hand side of the controller there are two slots that extend most of the height of the controller. The leftmost slot contains a collection of four small slender rectangular connectors. The slot to the right appears to be empty. There is an opened laptop computer positioned a little left of center of the slide. It is angled facing towards the right with a dark screen, keyboard and mousepad all visible. There is a dark cable connecting the right side of the laptop to one of the slender rectangular connectors on the controller. There is a label over the cable that says "CONSOLE CABLE.")

Graphics: Thinkstock (L) and Ralph W. Boaz (R)

 

Slide 74:

Preparations for Testing

USB Test Package Method

  • Preconfigured tests can be downloaded from the web to a USB flash drive
  • Simply plug the USB drive into the ATC unit and turn on the power
  • Variations (if desired) made by simple edits of the runAPIVS file on the USB drive
  • Windows or Linux environment

This slide has a graphic that shows a configuration used for testing API software. Please see the Extended Text Description below.

(Extended Text Description: This slide has a graphic that shows a configuration used for testing API software. The graphic is in the lower one-half of the slide. There is a transportation controller positioned a little left of center of the slide. It is about twice as wide as it is tall. It has two small keypads on the right side of the controller arranged vertically: the upper one has 4 rows by 3 columns of keys and the lower one has 4 rows by 4 columns of keys. There is a small screen in the center of the controller with the letters "ATC" over the top of it. There are four large circular connectors arranged horizontally below the screen. On the left-hand side of the controller there are two slots that extend most of the height of the controller. The leftmost slot contains a collection of four small slender rectangular connectors. The slot to the right appears to be empty. There is an opened laptop computer positioned a little right of center of the slide. It is angled facing towards the left with a dark screen, keyboard and mousepad all visible. Centered between the controller and the laptop is a USB flash drive (larger, not to scale of the other objects). Centered above the USB flash drive is a broad curved arrow pointing to the right representing the flow of information from the controller to the laptop. Centered below the USB flash drive is a broad curved arrow pointing to the left representing the flow of information from the laptop to the controller.)

Graphics: Ralph W. Boaz (L) and Thinkstock (R)

 

Slide 75:

Using the USB Test Package Method

  1. Download or clone the APIVS repository to a PC
  2. Install a USB flash drive into your PC
  3. Run package.sh (for Linux PCs) or package.bat (for Windows PCs) from a Linux or Windows shell, respectively
  4. Copy the VSE executable and APIVS Loopback Drivers to the USB drive
  5. (optional) Edit runAPIVS to modify tests
  6. Install the USB flash drive in the ATC unit
  7. Turn the ATC unit on
  8. Wait for completion
  9. Test results may be analyzed by reinstalling the USB flash drive on the PC and viewing the log files (*log.xml)

 

Slide 76:

Using the USB Test Package Method

Using the USB Test Package Method. Please see the Extended Text Description below.

(Extended Text Description: In the upper left-hand side of the slide is a cloud. The cloud contains the words "APIVS Repository." Below the cloud, covering the middle left of the slide, is a stick figure with a tie and holding a DVD. Beneath the stick figure is the text "Vendor Supplied VSE Executable." In the upper center of the slide is an opened laptop computer. It is angled facing towards the left with a dark screen, keyboard and mousepad all visible. In the upper right of the slide is a USB flash drive (larger, not to scale of the other objects). Below the USB flash drive, in the middle right of the slide is a transportation controller. It is about twice as wide as it is tall. It has two small keypads on the right side of the controller arranged vertically: the upper one has 4 rows by 3 columns of keys and the lower one has 4 rows by 4 columns of keys. There is a small screen in the center of the controller with the letters "ATC" over the top of it. There are four large circular connectors arranged horizontally below the screen. On the left-hand side of the controller there are two slots that extend most of the height of the controller. The leftmost slot contains a collection of four small slender rectangular connectors. The slot to the right appears to be empty. In the lower right of the slide, is another USB flash drive. In the lower center of the slide is another laptop. To the immediate left of the laptop is a stick figure with a smile and arms raised in the air. Next to the figure is the text "Happy Tester." There are curved arrows connecting the objects. There is an arrow extending from the APIVS Repository cloud to the laptop. It is labeled "Download or Clone." There is an arrow extending from the Vendor Supplied VSE Executable stick man to the laptop in the upper center of the slide. It is labeled "Copy Software." There is an arrow extending from the laptop in the upper center to the USB Flash drive in the upper right of the slide. It is labeled "Run package.sh or package.bat." There is an arrow extending from the USB Flash drive in the upper right of the slide to the transportation controller in the middle right of the slide. It is labeled "Install and Run." There is an arrow extending from the transportation controller to the USB Flash Drive in the lower right of the slide. It is labeled "Remove." There is an arrow extending from the USB in the lower right of the slide to the laptop in the lower center of the slide. It is labeled "Copy Results.")

 

Slide 77:

Prepare the APIVS Software for Testing

Command-Line Interface (CLI) of the APIVS Software

vse-L [1-3] [-c configuration-file] [-i APIVSXML-file] [-o output-file] [-n test_suite_name] [-R count] [-H] [-C]

Where:

  • vse - Name of the VSE executable program.
  • -L [1-3] - (required) Conformance level of the output desired.
  • -c configuration-file - (optional) File that specifies a series of VSE configurable items. If this file is omitted, default values are used.
  • -i APIVSXML-file - (optional) Path to the input XML test script to use. If -i is not present, the input will be read from stdin.
  • -o output-file - (optional) Path of where to place the generated output XML file. If -o is not present, the output will be placed on stdout.
  • -R count - (optional) Repeat test load count times, or indefinitely if count is 0.

 

Slide 78:

Prepare the APIVS Software for Testing

Editing runAPIVS (optional)

  • runAPIVS is a Linux shell script in the root of the USB Flash Drive
  • Defaults to running all tests cases on the API software one time with Logging Level 1
  • Easy edits to
    • Change the Logging Level
    • Increase iterations of particular tests
    • Run a subset of the test cases and/or change other VSE options

 

Slide 79:

Editing the runAPIVS Linux Shell Script

Editing the runAPIVS Linux Shell Script. Please see the Extended Text Description below.

(Extended Text Description: As the instructor discusses the lines "ln -s /dev/tnt0 /dev/sp6_loopback_a" and "ln -s /dev/tnt1 /dev/sp6_loopback_b," they are encompassed by rounded rectangles. As the discussion moves to the line "LEVEL=1," that line is encompassed by a rounded rectangle and the previous rounded rectangle disappears. The full text is below:

# Filename: runAPIVS
# File Type: Linux shell script
# Test Case: many
# Description: run VSE from USB at startup on
# specific test cases #
# Date Revision Description
# 2/24/16 1.0 initial release
# start async loopback driver; add symbolic links
insmod /media/sda1/APIVS/bin/tty0tty.ko
ln -s /dev/tnt0 /dev/sp6_loopback_a
ln -s /dev/tnt1 /dev/sp6_loopback_b
...
# set the conformance level this run (1,2,3)
LEVEL=1
---

)

 

Slide 80:

Editing the runAPIVS Linux Shell Script

Editing the runAPIVS Linux Shell Script. Please see the Extended Text Description below.

(Extended Text Description: As the instructor discusses VSE command line "vse -L $LEVEL -c ./VS_config_1.txt -i C2020_in.xml -o C2020_log.xml" elements of the line are encompassed by rounded rectangles as follows: "vse," "-L $LEVEL," "-c ./VS_config_1.txt," "-i C2020_in.xml" and "-o C2020_log.xml." As the discussion moves from element to element, the previous rounded rectangle disappears as the next one appears. The full text is below:

reset_modules
if [ "$DELETE_LOGS" == TRUE ]; then rm C2 02 0_log.xml; fi
clear_test_line; printf "Testing APIRI.TCS.2020... " >/dev/sp6
vse -L $LEVEL -c ./VS_config_1.txt -i C2020_in.xml
-o C2020_log.xml
print_test_result
reset_modules
if [ "$DELETE_LOGS" == TRUE ]; then rm C2 03 0_log.xml; fi
clear_test_line; printf "Testing APIRI.TCS.2030... " >/dev/sp6
vse -L $LEVEL -c ./VS_config_1.txt -i C2030_in.xml
-o C2030_log.xml
print_test_result
reset_modules
if [ "$DELETE_LOGS" == TRUE ]; then rm C2040_log.xml; fi
clear_test_line; printf "Testing APIRI.TCS.2040... " >/dev/sp6
vse -L $LEVEL -c ./VS_config_1.txt -i C2040_in.xml
-o C2040_log.xml
print_test_result

)

 

Slide 81:

Editing the runAPIVS Linux Shell Script

Editing the runAPIVS Linux Shell Script. Please see the Extended Text Description below.

(Extended Text Description: Author's relevant description: This slide is animated. As the instructor discusses editing the runAPIVS Linux Shell Script, various modifications occur and then are encompassed by rounded rectangles as follows:

  • "vse -L $LEVEL -c ./VS_config_1.txt -i C2020_in.xml -o C2020_log.xml" turns into "vse -L 3 -c ./VS_config_1.txt -i C2020_in.xml -o C2020_log.xml" with the "-L 3" encompassed by a rounded rectangle.
  • "vse -L $LEVEL -c ./VS_config_1.txt -i C2030_in.xml -o C2030_log.xml" turns into "vse -L $LEVEL -c ./VS_config_1.txt -i C2030_in.xml –R 10 -o C2030_log.xml" with the "-R 10" encompassed by a rounded rectangle.
  • "reset_modules if [ "$DELETE_LOGS" == TRUE ]; then rm C2040_log.xml; fi clear_test_line; printf "Testing APIRI.TCS.2040... " >/dev/sp6 vse -L $LEVEL -c ./VS_config_1.txt -i C2040_in.xml -o C2040_log.xml print_test_result" turns into # reset_modules # if [ "$DELETE_LOGS" == TRUE ]; then rm C2040_log.xml; fi # clear_test_line; printf "Testing APIRI.TCS.2040... " >/dev/sp6 # vse -L $LEVEL -c ./VS_config_1.txt -i C2040_in.xml -o C2040_log.xml # print_test_result with the "#" symbols encompassed by a rounded rectangle.

)

 

Slide 82:

Use the APIVS Software to Execute Tests

Running the USB Test Package

  • Plug the USB drive into the ATC and turn it on
  • Follow the screens that appear on the ATC unit front panel

 

This slide is a graphic illustrating the execution of the APIVS software when using the USB Test Package Method. Please see the Extended Text Description below.

(Extended Text Description: This slide is a graphic illustrating the execution of the APIVS software when using the USB Test Package Method. In the upper right of the slide is a square front panel of a transportation controller including a screen and keypads. Below the front panel is an enlargement of the screen. There are dotted lines extending from the screen in the controller to the edges of the enlargement. The enlarged screen is about 25% of the height of the slide and extends almost the entire width of the slide. The enlarged screen is 8 lines x 40 fixed width characters in size with the following lines of text visible:

ATC 5401 API Validation Suite v1.0

Begin Test [YES] / [NO]?)

 

Slide 83:

Use the APIVS Software to Execute Tests

Running the USB Test Package

  • Plug the USB drive into the ATC and turn it on
  • Follow the screens that appear on the ATC unit front panel

 

Use the APIVS Software to Execute Tests. Please see the Extended Text Description below.

(Extended Text Description: This slide is a graphic illustrating the execution of the APIVS software when using the USB Test Package Method. In the upper right of the slide is a square front panel of a transportation controller including a screen and keypads. Below the front panel is an enlargement of the screen. There are dotted lines extending from the screen in the controller to the edges of the enlargement. The enlarged screen is about 25% of the height of the slide and extends almost the entire width of the slide. The enlarged screen is 8 lines x 40 fixed width characters in size with the following two lines of text visible:

ATC 5401 API Validation Suite v1.0

Running test session.

Testing APIRI. TCS. 2010...

Test cases passed:13 failed:0)

 

Slide 84:

Use the APIVS Software to Execute Tests

Running the USB Test Package

  • Plug the USB drive into the ATC and turn it on
  • Follow the screens that appear on the ATC unit front panel

 

Use the APIVS Software to Execute Tests. Please see the Extended Text Description below.

(Extended Text Description: This slide is a graphic illustrating the execution of the APIVS software when using the USB Test Package Method. In the upper right of the slide is a square front panel of a transportation controller including a screen and keypads. Below the front panel is an enlargement of the screen. There are dotted lines extending from the screen in the controller to the edges of the enlargement. The enlarged screen is about 25% of the height of the slide and extends almost the entire width of the slide. The enlarged screen is 8 lines x 40 fixed width characters in size with the following two lines of text visible:

ATC 5401 API Validation Suite v1.0

Running test session.

Session complete.

Please remove USB drive and reboot.

Test cases passed: 40 failed: 0

)

 

Slide 85:

Activity. A placeholder graphic with an image of hand over a computer keyboard to show that an activity is taking place.

 

Slide 86:

Question

What is not an appropriate reason to edit the runAPIVS shell script?

Answer Choices

  1. Turn off all test output
  2. Change the number of iterations on a test
  3. Change the conformance report logging
  4. Select a subset of the existing test cases

 

Slide 87:

Review of Answers

A small graphical green and yellow check mark representing correct.a) Turn off all test output
Correct! One cannot turn off all test output. A pass/fail is the tersest output available.

A small graphical red and yellow X representing incorrect.b) Change the number of iterations on a test
Incorrect. Selecting a particular test case is a good reason to edit runAPIVS.

A small graphical red and yellow X representing incorrect.c) Change the conformance report logging
Incorrect. Changing the conformance report logging is a good reason to edit runAPIVS.

A small graphical red and yellow X representing incorrect.d) Select a subset of the existing test cases
Incorrect. Selecting a subset of the existing test cases is a good reason to edit runAPIVS.

 

Slide 88:

Learning Objectives

  • Explain the purpose of the API Validation Suite (APIVS) Software
  • Use the API Reference Implementation (APIRI) test documentation to specify acceptance testing
  • Use the APIVS Software to test the API Software
  • Interpret and report results of testing API Software

 

Slide 89:

Learning Objective 4

  • Interpret and report results of testing API Software

 

Slide 90:

Analyze Results Using Off-the-Shelf Tools

 

Slide 91:

Analyze Results Using Off-the-Shelf Tools

Analyze Results Using Off-the-Shelf Tools. Please see the Extended Text Description below.

(Extended Text Description: This slide is a snapshot of the Notepad++ editor. It is shown editing a file C2040_log.xml. There are four lines making up the header of the editor. The first line, starting at the left corner there is a small notepad and pencil icon followed by string "C:\Users\rboaz\Documents\PILLAR CONSULTING 2B\ITE\API Reference Implementation\Independent Testing\run161029_2100\C2040_log.xml – Notepad++." The second line of the header has the menu items listed horizontally "File," "Edit," "Search," "View," "Encoding," "Language," "Settings," "Macro," "Run," "Plugins," "Window" and "?." The third line of the editor has 31 small icons arranged horizontally. The fourth line of the header has height rectangles with filenames in them and an "X" that provide quick access to files that are currently in the editor.

The main body of the editor has a left boarder running vertically with line numbers 2-24. It takes up about 5% of the editor width. The text of the file shown is as follows:

2
3 <ApiVsRun date="2016-10-30 03:58:31 AM UTC" configuration="./VS_config_1.txt" input=
4
5
6 <TestSuite line="1320" timestamp="+0.147173" name="All_Test_Cases" description="
7
8 <TestCase line="1187" timestamp="+2.149510" name="Case_C2040" "description="C
9 <Sleep line="1188" timestamp="+2.149722" time="%5" />
10 <Call line="1200" timestamp="+7.150038" ref="fpui_open" >
11 <Function line="615" timestamp="+7.150131" funcName="fpui_open" retu
12 <Call line="626" timestamp="+7.150819" ref="Success_Handler" >
13 </Call>
14 </Call>
15 <Sleep line="1206" timestamp="+7.151509" time="%1" />
16 <Call line="1207" timestamp="+8.151738" ref="fpui_get_focus" >
17 <Function line="491" timestamp="+8.15183 funcName= "fpui_get_focus"
18 <Call line="502" timestamp="+8.153056" ref="Success_Handler" >
19 </Call>
20 </Call>
21 <Call line="1217" timestamp="+8.153195" ref="fpui_write" >
22 <Function line="1061" timestamp="+8.153271" funcName="fpui_write"
23 <Call line="1076" timestamp="+8.153548" ref="Success_Handler" >
24 </Call>

The slide is animated. As the instructor describes the logging that is output to a file during a test lines 9 through 13 are encompassed by a rounded rectangle.)

 

Slide 92:

Analyze Results Using Off-the-Shelf Tools

Analyze Results Using Off-the-Shelf Tools. Please see the Extended Text Description below.

(Extended Text Description: This slide is a snapshot of online tool XML diff. It is run in an internet browser. The URL for the website is https://www.corefiling.com/cgi-bin/xmldiff.cgi. Shown in the browser is a comparison of two output files from runs of the APIVS software. Since they are the exact same runs but at different times, the only differences are the date and timestamp lines. The "-" and "+" characters indicate the lines that were difference. The – lines are in red and the + lines are in green. Below is the output.

<?xml version="1.0" encoding="utf-8"?>
<ApiVsRun configuration='./VS_config_1.txt'
- date='2016-10-31 06:07:53 AM UTC'
+ date='1970-10-29 12:13:19 AM UTC'
input='./C2040_in.xml'
level='trace'
output='/tmp/C2040_log.xml'
testSuite='ALL_TESTS'
>
<Define
line='70'
- timestamp='+0.045790'
+ timestamp='+0.045831'
type='fpui_handle'
var='$fpui_handle'
/>
<Define
line='71'
- timestamp='+0.046147'
+ timestamp='+0.046198'
type='fpui_aux_handle'
var='$fpui_aux_handle'
/>

)

 

Slide 93:

Analyze Results Using Off-the-Shelf Tools

Analyze Results Using Off-the-Shelf Tools. Please see the Extended Text Description below.

(Extended Text Description: This slide is a snapshot of the XmlGrid online XML editor. It is run in an internet browser. The URL for the website is http://xmlgrid.net.

There is a blue banner across the top of the web page with the name in large elementary school looking print "XmlGrid.net." This is followed horizontally in smaller more traditional print "Home Demo FAQ." Beneath the XmlGrid.net are the words in italics "Online XML Editor." Beneath the blue banner there are rectangle menu buttons arranged horizontally. They are labeled "Input Box," "Open File," "By URL" and "Create XML". Below Input Box menu button is an icon that is a small arrow that is in the shape of a loop. To the right of the loop is a small icon of a binocular. To the right of the binocular icon are two additional rectangle menu buttons. They are labeled "TextView" and "Validate."

Starting from the left and below the menu buttons are the words "Data Source: ApiVSRun." To the right of this in smaller italics are the words "Well-Formed XML" followed by a checkmark. Below this is a set of three tables representing XML elements. Each table is preceded by the name of the element in a rectangular button as described below.

There is a button labeled "<> ApiVSRun." To the left of the button, there is a small arrow pointing downward and to the right. The table contains the following:
0 date 2016-10-30 03:58:31 AM UTC
@configuration ./VS_config_l.txt
@ input ./C2040_in.xml
@ output /tmp/C2040_log.xml
@ testSuite All_TESTS
@ level summary

There is a button labeled "<> TestSuite." To the left of the button, there is a small arrow pointing downward and to the right. The table contains the following:
@ line 1320
@ timestamp +0.147173
@ name All_Test_Cases
@description Run All Test Cases

Below the table is a rectangular button labeled "<> TestCase." To the left of the button, there is a small arrow pointing to the right. There is no table beneath this button.
There is a button is labeled "<> RunResult." To the left of the button, there is a small arrow pointing downward and to the right. The table contains the following:
@date 2016-10-30 03:58:47 AM UTC
@status PASS

The TestSuite and RunResult buttons and tables are indented further from the right than the ApiVSRun button and table. The TestCase button beneath the TestSuite button is indented further to the right.

There is a thin green line connecting the ApiVSRun button to the left side of the RunResult button. There is a thin green line connecting the TestSuite button to the left side of the TestCase button. There is a thin purple boarder encompassing all of the items below the Data Source line.)

 

Slide 94:

Analyze Results Using Off-the-Shelf Tools

Analyze Results Using Off-the-Shelf Tools. Please see the Extended Text Description below.

(Extended Text Description: This slide is a snapshot of online tool XML Viewer online XML editor. It is run in an internet browser. The URL for the website is https://codebeautify.org/xmlviewer.
There is a banner across the top of the web page. At the top left of the banner is a small outline figure of a human head with what appears to be a brain. To right of the figure in large bold type is the website name "Code Beautify." To the right of the website name is a set of menu commands separated by vertical bar characters as follows: "HTTPS | JSONFormatting.org | My Ip | Search | Recent Links | Sample | More | Sign in | (?)."

Below the banner on the left side of the webpage in large type is "XML VIEWER." On the right side of the webpage is a rectangular button labeled "Save & Share." Below XML VIEWER line is a rectangular area with the text "Result: XML Tree View" centered within the area. On the right side of the rectangular area are two small icons; one that looks like two pieces of paper (used for copying) and the other that has four arrows extending outwards (used for changing to full screen view). The remainder of the website shows the elements of the XML file in a tree form with four XML elements as described below.

The upper left of element has a dash symbol next to an orange rectangle with white text that says "apivsrun." The text below the element is as follows:

@date: 2016-10-30 03:58:31 AM UTC
@configuration: .VS_config_txt
@input: ./C2040_in.xml
@output: /tmp/C2040_log.xml
@testSuite: ALL_TESTS
@level: summary

The next element is indented to the right of the indentation level of the apivsrun element. There is a dash symbol next to an orange rectangle with white text that says "testsuite." The text below the element is as follows:

@line: 1320
@timestamp: +0.147173
@name: All_Test_Cases
@description: Run All Test Cases

The next element is indented to the right of the indentation level of the testsuite element. There is a dash symbol next to an orange rectangle with white text that says "testcase." The text below the element is as follows:

@line: 1187
@timestamp: +2.149510
@name: Case_C2040
@description: 02040: FPUI Reading and Writing Data

The next element is indented to the right of the indentation level of the testcase element. There is a dash symbol next to an orange rectangle with white text that says "sleep." The text below the element is as follows:

@line: 1188
@timestamp: +2.149722
@time: %5

)

 

Slide 95:

Create Test Reports Using the APIVS

Conformance report options

This slide has a picture of a stack of paper on the left-hand side of the slide

Graphic: Thinkstock

  • Testers may include test logs in their test reports
  • Level 1 - Conformance/nonconformance indication only
    • 304 lines of output - about 16 minutes
  • Level 2 - Conformance/nonconformance indication and summary result
    • 9,693 lines of output - about 16 minutes
  • Level 3 - Conformance/nonconformance indication, summary result, and all logs and traces
    • 73,066 lines of output - about 22 minutes

 

Slide 96:

Create Test Reports for the API Software

IEEE 829-2008 Level Test Report Outline

1 - Introduction

1.1 - Document identifier

1.2 - Scope

1.3 - References

2 - Details

2.1 - Overview of test results

2.2 - Detailed test results

2.3 - Rationale for decisions

2.4 - Conclusions and recommendations

3 - General

3.1 - Glossary

3.2 - Document change procedures and history

 

This slide has a picture of a stack of paper on the right-hand side of the slide.

Graphic: Thinkstock

 

Slide 97:

Combine All of the Out Log Files into the Detailed Test Results

2.2 Detailed Test Results

...

  • <APIVSRun date="2016-10-30 04:48:07 AM UTC" configuration="./VS_config_1.txt" input="./C1310_in.xmr output="/tmp/C1310_log.xml" testSuite="ALL_TESTS" level="conformance" >
    <RunResult date="2016-10-30 04:48:17 AM UTC" status="PASS" />
  • < APIVSRun date="2016-10-30 04:48:18 AM UTC" configuration="./VS_config_1.txt" input="./C1320_in.xml" output="/tmp/C1320_log.xml" testSuite="ALL_TESTS" level="conformance" >
    <RunResult date="2016-10-30 04:48:29 AM UTC" status="PASS" />
  • < APIVSRun date="2016-10-30 04:48:30 AM UTC" configuration="./VS_config_1.txt" input="./C1330_in.xml" output="/tmp/C1330_log.xml" testSuite="ALL_TESTS" level="conformance" >
    <RunResult date="2016-10-30 04:48:41 AM UTC" status="PASS" />

...

 

Slide 98:

Activity. A placeholder graphic with an image of hand over a computer keyboard to show that an activity is taking place.

 

Slide 99:

Question

True or False: It is a good idea to always log as much information as possible on all tests.

Answer Choices

  1. True
  2. False

 

Slide 100:

Review of Answers

A small graphical red and yellow X representing incorrect.a) True
Incorrect. If a tester wants to view full logging, it is better to do this on selected tests.

A small graphical green and yellow check mark representing correct.b) False
Correct! Logging all of the data creates a voluminous report and makes understanding the results difficult. While full logging on all tests can be done, it is advised that testers repeat their test with full logging for tests that failed previously.

 

Slide 101:

Module Summary

  • Explain the purpose of the API Validation Suite (APIVS) Software
  • Use the API Reference Implementation (APIRI) test documentation to specify acceptance testing
  • Use the APIVS Software to test the API Software
  • Interpret and report results of testing API Software

 

Slide 102:

ATC Curriculum Completed To Date

A small graphical green and yellow check mark representing correct.Module A207a/b:
Building an ITS Infrastructure Based on the Advanced Transportation Controller (ATC) 5201 Standard

A small graphical green and yellow check mark representing correct.Module A208:
Using the ATC 5401 API Standard to Leverage ITS Infrastructures

A small graphical green and yellow check mark representing correct.Module A307a:
Understanding User Needs for Advanced Transportation Controllers (ATC) Based on ATC 5201 Standard v06

A small graphical green and yellow check mark representing correct.Module A307b:
Understanding Requirements for Advanced Transportation Controllers (ATC) Based on ATC 5201 Standard v06

A small graphical green and yellow check mark representing correct.Module T307:
Applying Your Test Plan to the Advanced Transportation Controller (ATC) Based on ATC 5201 Standard v06

A small graphical green and yellow check mark representing correct.Module T308:
Acceptance Testing
for Advanced Transportation Controller Application Programming Interface Software

 

Slide 103:

Thank you for completing this module.

Feedback

Please use the Feedback link below to provide us with your thoughts and comments about the value of the training.

Thank you!