After just under a year of study and expert testimony, the National Academy of Sciences panel found that members believe there is a better way for federal regulators to separate safe from high risk motor carriers. This was known as the Item Response Theory, which was established in 2017.
Fast forward six years, after research and evaluation by the Federal Motor Carrier Safety Administration to dismiss the complex data-driven method for one motive: the FMCSA believes it is too complex for the truckers to understand. The FMCSA is proposing to make improvements in the safety measurement system. This system will be based on percentile ratings guided by the Compliance, Safety, Accountability system (CSA). The CSA has had mixed reviews by motor carriers after it was instituted in 2010.
After early reviews of early reactions from different transportation, stakeholders show that a few in the industry agree with the regulators. However, some data experts, including the members of academic panel and trucking data enthusiast, Steve Bryan, believe the IRT method’s rejection is a long lost opportunity. The agency decision however, according to Brandon Wiseman (safety executive), had blocked the adoption of a system that had been a potential “monster” for trucking companies.
Dan Horvath, Vice President of safety policy for American Trucking Associations, believes that the IRT shows promise in identifying carriers for crash risk. However, Horvath points out that details matter and the FMCSA has identified a few issues with the IRT, primarily, its complexity. Horvath believes that it is important to explain the methodology in a simple way, while the FMCSA believes it is not possible.
Sean Garney believes the IRT has a lot of potential from a truly academic perspective. Garney is the co-director at Scopelitis Transportation Consulting. However, Garney believes the FMCSA made the correct choice as it requires a lot of data, some of which the FMCSA could not get. Furthermore, he feels it is very hard to understand, and they are already having challenges describing the CSA.
Three members of the Academy of Sciences stated after studying the past rating system of the FMCSA, that the tweaks to the SMS system a “positive step”. The three members are Joel Greenhouse, Sharon-Lise Normand, and Michael Cohen. These members believe that by not fully implementing the IRT approach, the FMCSA will not benefit from the structure the IRT model provides. This model systematically combines the individual data analytic data, which inputs into a system, accounts for accumulated unreliability in the data, and even produces scientifically based decision-making.
It has been reported that the models have been successfully executed in complex settings like in health care and education. Without using the IRT modeling in-house, the FMCSA will not benefit from background and experience of experts who have fortuitously developed models and understand both the implementation and computational challenges in the IRT models in practice.
Committee member and program co-director of the Upper Great Plains Transportation Institute at North Dakota State University, Brenda Lantz, notes the practical challenges listed in the IRT, particularly how well the general public can understand the writing of the program. She is pleased to see that they were able to gain insights during the testing of the IRT, which helped lead to proposed changes that will improve the SMS.
The CEO of Bluewire was actually a fan of the IRT as it was an excellent solution. Bryan believes that some people at FMCSA did not understand the science behind the IRT, so they were unable to explain it to the truckers. Bryan notes the science behind credit scores, IQ scores, or how educators compute grades. Each of those are IRT models, and everyone trusts each model as it was good science.
Overall, the IRT has had mixed reviews by people within the trucking industry. There are those who believe it is far too complex for people to understand. Others believe that the IRT is helpful even though it is complex. Thankfully, the feedback from the trucking industry will help the FMCSA make improvements to the IRT.
If you are in need of moving equipment like bulldozers, a prized Lamborghini, or a beat up clunker, we at SAC are more than capable of handling and walking you through the entire process. If you are interested, contact a Ship A Car representative today to get started!
