T3 Webinar Question and Answer Transcript

Keeping Risks in Check: Applying the Updated FHWA Model Systems Engineering Document to Adaptive Signal Control Technology Systems (ASCT) Implementation (December 20, 2012)

Back to Webinar Files

Q. I wonder if the FHWA interest in adaptive control is translated into funding to support the data collection and maintenance to support adaptive control?

Rick Denney: The funding for doing adaptive control projects is the same funding we've always had, which can be applied to that, through the Congestion Mitigation and Air Quality Improvement (CMAQ) Program and the Surface Transportation Program (STP). I know everybody wishes that we had something really specific to target to it, but that's the answer.

Q. If a municipality is installing a second system with similar needs to the first system, in just a different location, they would be able to justify synchronization with the existing system?

Rick Denney: Yes, and there's a couple of different ways of approaching that. Certainly if the two systems are integrated together, synchronization would be easy to do. Now what is not easy to do is to start with an experimental application and then try to expand that into a much larger implementation using synchronization. That ends up being a dodge around the whole requirement for justifying a sole source acquisition, and that will usually run into trouble once it has to be approved. So we would recommend that synchronizing to an existing system can certainly be done, but we would expect that the existing system was fully supported by a systems engineering analysis. And there's a second aspect to that, too though, and that is that there's other strategies for using the same systems engineering effort across multiple implementations, and if an agency has an overall program of building adaptive control on a number of arterials and all of those arterials share the same needs and requirements, and there's certainly nothing wrong with developing a systems engineering documentation for that program of projects and then applying it equally to all of those projects. It's also possible to apply the systems engineering documents on an even larger scale, and some of our FHWA divisions have worked with state agencies to develop approaches using programmatic agreements to govern the application of one set of systems engineering documents to a class of projects. And when they do that, all we need is some demonstration that the next project in question is a bona fide member of that class, and the way we test that is, are all the needs and requirements as documented, a complete and correct compilation of needs and requirements for this new project. And if the answer to that is yes, then most of our divisions will not require that the system engineering be done over again. And then there is the issue of agencies having installed one system and they want to install another system that works the same way, uses the same sort of equipment, so that their technicians don't have to learn a new system. And you know, that one's a little tougher, and we're sort of having an internal conversation about how that might be handled. It could certainly be handled as a programmatic or a program wide acquisition where you buy one system to be installed in a number of places, and you're paying for the licensing of that system to a number of places in the first project. That's one approach to that. But it might also be an application for a public interest finding, where we say that, even though the operational requirements don't really justify a sole source acquisition for that second system, our maintenance needs do. And those maintenance needs could certainly be articulated as regular requirements but it might be a reason why application of a public interest finding, and again, we're having that conversation internally, and we don't really know how that's all going to shake out at this point.

Q. If the agency has a valid public interest finding for a specific technology, what procedures must the agency do to meet federal funding requirements? Which of the seven system engineering requirements must the agency document?

Rick Denney: In fact, all seven, simply because there is nothing about section 940 of the Code of Federal Regulations, that is set aside by fulfilling the requirements of section 635 in the Code of Federal Regulations. So meeting the requirements for a sole sourced acquisition does not actually address the requirements for doing systems engineering. However, doing the systems engineering does give you the tools you need to justify the sole source acquisition as required by section 635.

Paul R. Olson: When we talk about looking at requirements, the people that write them have to have a good idea of how they want them to be addressed, and so when you're looking at vendors, say, response to your requirements, you've got to know something about it in order to be able to judge whether or not they're meeting our needs, so it's not a rote thing. You've got to really understand what you need and what you're looking for to be able to evaluate those. That's why we say, yes or no answers in response to requirements is probably not a good thing.

Rick Denney: Yeah, and the other thing is, you'll know a whole lot more about adaptive controls, about what your needs are, regarding adaptive control, by the time you have a set of requirements in hand, if you've really dug into them the way we recommend. And if so, then it will empower you to ask very direct questions and get very direct answers, and know how to evaluate those answers when you get them.

Q. Has “best value” been successfully used? The DOT and FHWA has indicated only “low bid” can be used for construction.

Paul R. Olson: And that—it's not true that you can only do low bid for these kinds of things. In fact, we've got Special Experimental Project 14 (SEP 14), which we rolled over, I think into some other regulations that allow you to do best value and we'd recommend that, for this, that's probably the best way to go.

Rick Denney: Yes, as represented by that RFP-based process that I talked about at the top of my diagram.

Q. Is ACS Lite commercial-off-the-shelf (COTS)?

Rick Denney: I think that only one manufacturer offers ACS Lite as a product at all, but as I said in my presentation, there are no adaptive systems that would meet the Federal acquisition regulation definition of commercial off the shelf.

Q. Who do we work with to do best value? I don't think our state knows or understands this. It probably needs to trickle down and be discussed.

Rick Denney: Yes, you're—The statement is, maybe the folks in our state don't understand best value stuff, and boy, that's absolutely true. And we have found that, in some states, where the states are handling oversight requirements for Federally funded projects, the states themselves are struggling with this concept, and in some cases, our division offices are new to it as well. So this is an effort that we're making, FHWA is not monolithic, and state DOTs are not monolithic by any means, so it's some—that's one of the reasons why we hold these webinars and try to spread this information out as much as we can.

Paul R. Olson: There are some states, in fact, that state division, Federal Highway Administration Division operations folks and get them in the conversation.

Rick Denney: There are some states that are legally—that—constrained to certain acquisition approaches and just flat won't allow an RFP-based approach no matter what. And that's why we include a description of how requirements feed, and standard design-bid-build process, too, because either system, either approach works better if you have the requirements going into the process. So—

Q. What would you say the top three requirements and/or expectations and/or lesson learned by the various agencies looking for adaptive systems?

Paul R. Olson: You know, maybe the top one is, is that agencies have a hard time describing the situation or the problem they're attempting to solve. The other one is that a lot of agencies, once you start going through this effort, find out, well gee whiz, I already got what I need to solve my problem, I don't need to spend the money on adaptive and that maybe we should completely work through those before we go and look further at adaptive.

Rick Denney: And one of the things we've also seen is, is that even after the traffic guys in an agency put in a tremendous amount of effort, going through this process, then it gets turned over to the contracts people and they throw all that away and use their regular way of doing it. So that's a challenge for us to try to achieve, you know, a really comprehensive application of systems engineering as part of the procurement process. That's why we started talking about procurement a lot more than we once did.

Paul R. Olson: But we see a lot of people artificially constraining themselves to things like, it's got to work with all existing infrastructure. It can't require any additional detection, can't require any additional communications, and once you start writing those kinds of constraints, then your system may not do what you think it's going to do.

Q. Is there any documentation on before and after studies for adaptive?

Rick Denney: I do have some data that I can report, so let me summarize that real quickly. Deployment data for systems installed between 2010 and 2012, approximately 63 systems. We don't have much data on deactivation, but we know of some. That is anecdotal, but it worries us. Frankly, it would worry us if it was ten percent. So that's about the strongest statement we can make. That probably—that answer is probably somewhere between how I characterized it before, and how the question was asked, so there you go.

Paul R. Olson: A part of those failures is that they were early implementations, and none of them had any systems engineering to start with, and so it wasn't clear to anyone whether the people that purchased them actually understood what they needed or what the system would do to fulfill those needs. A lot of them were experimental applications as well.

Rick Denney: Yes. We certainly anticipate that as time goes on, that statistic will go way down, and we hope that these model systems engineering documents will facilitate that movement. We've also seen significant increase in the use of systems engineering, not always using our model documents. We figure something approaching two-thirds of projects built in the last two or three years have gone in with some kind of systems engineering effort, so that's why we expect the satisfaction rate to improve a lot.

Q. Any thoughts on the cost of conducting a SE analysis? Is there a rough range of costs?

Rick Denney: We have some data on this. One agency out west conducted it from the perspective of being an actual beginner. They had never done this before, and they put in, with their own staff time, about three and a half weeks. So let's call that, what, 140 hours? And the document they produced actually looks real good. I've worked with agencies and done it in less time, but we had sort of an advantage going in, because I was already very familiar with the model documents.

Paul R. Olson: Now that agency says about their work, that the majority of the time was making sure that they understood what the existing situation was, and what the problem was—defining the problem that the adaptive system needed to address.

Rick Denney: And in that process, they decided they needed to improve some of their regular operation. They did not include that. So the three and a half weeks was the effort to generate the documents. So hopefully that will give you an idea.

Back to Top