Transport data management with City Science

7th Apr 2020

Applications of software engineering best practice to transport

Get ahead with CIHT Membership

Join other savvy professionals just like you at CIHT.  We are  committed to fulfilling your professional development needs throughout your career

Find out more

By Alex Dawn and Laurence Oakes-Ash, City Science  

This blog has been produced with City Science, a CIHT Knowledge Partner.

This blog entry is a white paper extract from City Science, a CIHT knowledge partner. The views expressed do not necessarily reflect CIHT policy position. Click here to read the full paper.

This post is part if series of blogs produced with our CIHT Partnership Network.

Opportunities can be afforded to the transportation industry from the adoption of techniques, practices and data processing methods established within the software industry as ‘best practice’.

Developing software is a complex undertaking that often involves whole teams of people.  At the same time, software failure is expensive, with numerous high-profile examples of implementation projects running over budget, being delayed or being shelved entirely due to functionality not being delivered. Given the requirement to reduce risk and manage software delivery projects, the software industry has invested considerable effort to reduce the cost of development and improve software quality.  Overall, these best practices are shown to reduce the risk and life-time costs of software projects. 

Transport modelling exercises share many similar features with large-scale software development projects. For example, transport modelling projects often require:

  • Collaboration across large expert teams;
  • Management of complex model structures and data flows;
  • Detailed expertise including data processing and coding;
  • Multiple versions and scenarios to be managed simultaneously;
  • Continuous tracking and audit to ensure adherence with user expectations of quality (e.g. DfT WebTAG guidance).

In addition, the world of modelling is changing with growing expectations of what models will be able to do.  For many authorities, this includes the potential to link transport models with wider transport datasets (including real-time datasets) and use models for a range of purposes beyond traditional ‘static’ strategic scheme appraisals.  

Potential benefits can be afforded to transport modelling through the adoption of software engineering best practice. Overall, we conclude there are considerable opportunities to utilise these best practices to deliver three key benefits:

  • Quality – Using best practice to strengthen quality assurance, improve model transparency and reduce risk;
  • Collaboration – Making best use of existing tools to enhance collaboration within teams, with external partners and stakeholders; and
  • Efficiency – Using best practice to streamline processes and improve performance and productivity.

The Cost & Quality Challenge

While the software is among the most widely used products in human history, it has one of the highest failure rates of any product due to the complexity of developing software of sufficient quality (Capers, 2011).  Software failure is expensive, with numerous high-profile examples of implementation projects running over budget, being delayed or being shelved entirely due to functionality not being delivered.  The cost of poor software quality has been shown to increase as complexity increases.  The analysis shows that for the most complex projects, low quality in software development can increase costs by a third (Capers, 2011).  Even for the simplest projects, low-quality software development can increase overall costs by 16%. The costs of fixing flaws in software projects have been shown to increase exponentially the further the project progresses without the flaw being identified.

Reducing the cost of software development and improving software quality has therefore been critical objectives for the software development industry to reduce risk and manage delivery projects. 

Transport Modelling – a Complex Software Project

Transport modelling shares many similarities with complex software development.  It is common for transport model development to require a number of components to be brought together based on the needs of the users.  Detailed network coding and data processing are then required to develop the models.  Through this process, the modelling team will need to deal with a range of data and work collaboratively across the model while ensuring quality. 

Similarities include the following:

  • The need for collaboration across large expert teams.
  • Management of complex model structures and data flows. 
  • Detailed expertise including data processing and coding.
  • Multiple versions and scenarios to be managed simultaneously. 
  • The need to ensure adherence with user quality expectations.
  • Increasing requirement to engineer solutions around the core model. 

The software engineering industry has developed a range of best practices that can address these issues. There are considerable opportunities to utilise software engineering best practices to deliver three key benefits to transport modelling:

  • Improved Quality Assurance – Using best practice to strengthen quality assurance, improve model transparency and reduce risk;
  • Enhanced Collaboration – Making best use of existing tools to enhance collaboration within teams, with external partners and stakeholders; and
  • Streamlined Efficiency – Using best practice to streamline processes and improve performance and productivity.

Quality

Managing data and model versions

Transport authorities and their modelling teams want to ensure that versions of models can be tracked alongside their input data and assumptions throughout the life of the build and operation of the model.  In software engineering, best practice is to use a system of automated version control to manage changes.  This technique can be applied to transport modelling with a range of benefits.

One of the key ways that errors can be detected and model owners can be assured of the authority of their models is through automated change-tracking and automated audit/changelogs. 

Understanding the Design Rationale for Model Components

In a version control system, every change is accompanied by a ‘commit message’ which documents what’s changed and why.  This means the project benefits from having a traceable description of all changes. 

There is a range of areas in transport modelling where this is likely to be beneficial.  For example, WebTAG states the following regarding network development:  “Documentation of network coding assumptions is essential... Whilst it is recognised that assumptions can be changed (during calibration and validation), the need to record a consistent and sensible starting set of assumptions is important.”  Version control allows the development team to consistently document assumptions underpinning every change that is made.

Managing spreadsheet errors

Research has shown that in general within spreadsheets, errors occur in a few percents of all cells, meaning that for large spreadsheets, there is almost near certainty that error exists somewhere (Panko, 2008). 

A range of transport model development processes currently relies on spreadsheets which play key roles around traditional modelling software.  Within systems such as Excel, managing consistency is difficult due to the potential for users to make manual changes. Using an extensible version control system, however, process and model development can be managed in the same way.  A so-called ‘pipeline’ can be used to provide testing around all the end-to-end components of the transport model, reducing the risk of error while at the same time allowing for greater flexibility. 

Integrating version control, validation and error checking support into the transport model development process can provide the following benefits:

  • Reduced risk of error from spreadsheet-based systems;
  • Consolidated data, providing a useful and valuable asset in its own right;
  • Improved audit trail around changes introduced through the calibration process;
  • Improved documentation, going beyond documentation specifically for the LMVR;
  • The ability to manage a consistent standard and approach across multiple suppliers;
  • Future ability to automate data updates and production of downstream reports.

Collaboration

A range of barriers exists within existing transport software systems which limit the degree to which users can share models and their data.  This impedes both collaboration and transparency. 

Overcoming Knowledge Barriers through Enhanced Visibility

These range from visual systems to understand software components and dataflows, visual systems to enable individual users to identify issues more easily, systems to share workflow and systems to enable teams to collaborate on complex projects.

Version control can act as a backbone, increasing visibility of the work being conducted and enhancing the transparency of the process.  Beyond this, in a transport modelling context, visualisation tools can then be used to enable consistent understanding, improved communication and greater transparency. 

Overcoming Cost Barriers through Web-Based Sharing

Through web-based tools, the ability to share information between teams increases materially.  Cloud technologies are breaking down barriers within and between organisations by delivering services and applications that can be accessed from any device.  Reduced deployment costs and multiple users are reducing the barriers of traditional software.  This, in turn, enables more users to view the model and increases transparency. 

Enabling Self-Service to Overcome Access Barriers

A modern version control system, with appropriate adaptations for transport models, can provide self-service for authorised consultants, saving time and cost transferring these.  This makes it much easier for a client to view their models.  An online, version-controlled system that allows users to self-serve model components can also facilitate improved use of the model in downstream applications. 

Efficiency

One of the challenges levied in transport modelling is the cost and time taken to build and refresh models.  The introduction of software engineering best practice could enable considerable automation of workflows through the use of ‘pipeline’s, reducing risk and improving efficiency. A pipeline is used to combine a version control system, a testing framework, and the processes used to create the end deliverable (Merron, 2018). In a transport modelling context, this would be running and producing the model outputs and analysis as an end-to-end process from the underlying datasets. 

In the short-term, setting up projects as a pipeline could enable changes discovered later in the process to be incorporated much more easily.  Over the longer-term end-to-end pipelines could enable much more frequent update cycles, going beyond the current frequency of 5-yearly updates, potentially at a reduced cost.  Time and resource spent producing the model could be reinvested into the use of the model and development of a wider range of scenarios and what-if analyses.

Using Version Control to Streamline Changes and Reduce Build Effort

Traditionally, if a change is made to the base year model, every other model would need to be changed manually.  Using a version control system, changes can be fed through to all the scenarios digitally saving time and effort making the bank of models much easier to manage.

Streamlining Data Processing

Standardised data processing can be developed to process raw data prior to use within the model.  For example, count point data could be standardised enabling it to be automatically converted into an appropriate format for viewing and analysis alongside the transport model. 

Calibration & Validation

Calibration and validation are widely considered one of the most time-consuming and risky parts of model development.  According to TfL guidance “Calibration can be a lengthy and time-consuming exercise.”  Using pipelines, automated processes can be built around calibration and validation to accelerate the process while reducing risk. 

Streamlining Downstream Processes

Finally, transport models often support a number of downstream tasks such as economic modelling, air quality modelling or reporting.  By adopting software engineering best practice, an increasing number of downstream processes could also be brought into an end-to-end pipeline, essentially eliminating manual interfaces and updating these outputs automatically whenever input data changes. 

Conclusion

Transport modelling is changing.  Driven by an explosion in data; the growing complexity of models; changing demands of customers and the public; an increasing desire to link strategic and real-time models; and the need for more granular predictions through time and space, transport modelling practitioners are having to reassess the traditional tools.  Robust data management and version control is a fundamental pre-requisite to support clients in this new world.  But embracing these new technologies does not have to be costly. Quite the contrary – new technology, developed from software engineering best practices will reduce risk and cost while creating better outcomes for clients.  By improving quality assurance, enhancing collaboration and streamlining efficiency, data management tools will help you maximise value from your modelling activities.

 

Click here to read the full City Science white paper.

Comments on this site are moderated. Please allow up to 24 hours for your comment to be published on this site. Thank you for adding your comment.
{{comments.length}}CommentComments
{{item.AuthorName}}

{{item.AuthorName}} {{item.AuthorName}} says on {{item.DateFormattedString}}:

Share
Bookmark

Get ahead with CIHT Membership

Join other savvy professionals just like you at CIHT.  We are  committed to fulfilling your professional development needs throughout your career

Find out more