In a May 30 filing with the Postal Regulatory Commission, the Postal Service announced changes to its service measurement process. As stated in the filing:
“In this filing, the Postal Service provides notice to the Commission about revisions to the SPM Plan and Methodology Documents related to necessary changes to allow the Postal Service to collect and report service performance data at the 5-Digit ZIP Code level. These changes are consistent with and supportive of the operational initiatives and transition to 5-Digit to 5-Digit ZIP Code service standards proposed by the Postal Service in Docket No. N2024-1, and the Postal Service’s related intentions to implement updates to service performance measurement that will allow the Postal Service to generate service performance results at the 5-Digit to 5-Digit ZIP Code pair level. …
“The Postal Service intends to implement a two-phased approach to enhance the SPM system to enable service performance measurement at the 5-Digit ZIP Code level. This will entail revisions to the methodologies for data gathering and reporting in Legs 1 and 3. These changes will, among other things, enhance our public-facing Dashboard by providing more granular data for public review.
“To enable service performance measurement at the 5-Digit ZIP Code level, the Postal Service will leverage a cluster sampling approach in conjunction with CPMS scans, census package data, and geo-location breadcrumbs to improve both the measurement capabilities and statistical controls and predictability for the sampling process. CPMS scans, which are performed daily on over 99 percent of collection boxes, will provide a quasi-census measure for Leg 1 measurement by identifying when a carrier arrives at a collection box and collects the mailpieces. The CPMS scans will then be validated using a geo-fence, ensuring that collection box scans occurred at the physical collection box location. The geo-fence logic will also measure the arrival of carriers at the postal facility. Similarly, package delivery scans will be used in concert with mail delivery samples to validate Leg 3 measurement.”
Under the Postal Service’s new network model, “Leg 1” is from mail deposit to the origin Regional Processing and Distribution Center. “Leg 2” is between the origin and destination RPDCs, and “Leg 3” is from the destination RPDC to the addressee.
The agency further explained its “cluster sampling” concept:
“Cluster sampling is a statistical sampling method that allows large data populations to be divided into smaller ‘clusters’ with consistent features – such as geographic regions – where the overarching cluster sample results can be applied to the underlying elements of the cluster. The SPM system will leverage clusters based on regions, facilities, and volume density to ensure statistically significant sampling and reporting can be done at the 5-Digit ZIP Code level moving forward. Cluster sampling is a highly effective and cost-effective approach for sampling and reporting at the 5-Digit ZIP Code level. In the circumstances where 5-Digit ZIP Codes have volume below statistical threshold – low volume density for geographical regions and force majeure incidents – performance will be imputed from the density cluster that encompasses the 5-Digit Zip Code. This approach will provide statistically significant and representative measurement for these 5-Digit ZIP Codes.
“Cluster sampling offers cost and time efficiency. Controlled random sampling performed within the cluster is predictive for all elements of the cluster, which will allow the Postal Service to impute data from clusters when individual reporting elements do not meet sample minimums, reducing overall number of samples as well as associated costs. Cluster sampling is also scalable. Appropriately designed clusters with a controlled sampling process are usable for both elements within the cluster as well as overarching higher-level clusters. Moreover, cluster sampling offers predictability given that sampling from well-constructed clusters enhances the consistency and interpretability of results.”
The USPS added that the first phase, involving “Leg 3,” will be implemented on July 1, and the second phase, regarding “Leg 1,” will be effective September 1.
Observations
As might be expected, the filing is rife with positivity about how the new process will “measure service performance in an accurate, reliable, and representative manner.” As regular observers of the USPS and readers of its official statements know, former postmaster general Louis DeJoy’s acolytes – who are still in USPS HQ and executing his Plan – always described whatever they were doing or proposing in glowing terms guaranteeing success and great results.
Similarly, such statements were often long on generalities and short on specifics – and the proposed changes to SPM, further padded by statistical buzzwords, are no exception.
One telling reference is to “the operational initiatives … proposed by the Postal Service in Docket No. N2024-1.” That docket proffered a measurement scenario based around the new network model and the deployment of Regional Transportation Optimization, which ended afternoon collections at post offices more than 50 miles away from an RPDC – more than 70% of the nation’s ZIP code areas.
In presenting the proposed changes and the “cluster sampling” it will employ, the USPS is silent on how the “Day 0” it assigns to mail from RTO-impacted office will be reflected in its measurement process. Regardless of whether a carrier scans in collection mail or its arrival at the serving post office, if that mail involved is assigned “Day 0” it’s effectively deferred from inclusion in service performance calculations until it’s picked up the following business day. Perhaps the homogenization inherent in “cluster sampling” is expected to allow the better service for customers of non-RTO offices to obscure the true service rendered to the relatively smaller number of customers at RTO-offices, in turn yielding numbers supportive of upbeat service claims.
Similarly unexplained is how measurement will work if the agency’s model (above) treats the travel from the destination plant to delivery as another “Day 0.”
Fortunately for the USPS, the popular media and most customers don’t look behind its claims of service to see how selective measurement practices can be used to misrepresent reality. Commercial mail producers and their clients should not be so naïve: the Postal Service isn’t going to build a service measurement system that it isn’t sure will yield favorable results; those results should be evaluated accordingly.