Agencies submitted technology requests every year, and those requests had to become a funded list. Before the matrix, a request’s fate depended on how well it was argued, who argued it, and when it landed.
Two things followed from that, and both were problems. There was no way to tell an agency why its project had placed below another’s. And there was no way to defend the resulting list to the administration or to the Council.
A scored form lets two projects be set beside each other and the difference between them pointed at, line by line.
Built in March 2008 as Technology Officer under the interim Chief Information Officer: thirteen criteria in five groups, covering budget, efficiency, mandate and service.
The weights are the funding policy
They were set in writing before any particular project was scored, which is the whole of the design.
Legal and regulatory mandate and the level of administrative sponsorship carried the most. Loss of existing functionality sat at two and a half — so keeping what already worked outranked every efficiency or improvement argument on the sheet. Return on investment, which a private sector version of the same form would put first, sat in the bottom group at one and a half alongside community impact. And the lowest weight in the instrument went to projects required by a reorganization, which quietly declined to let internal restructuring jump the queue.
Deciding in advance that a legal mandate counts for three and a reorganization counts for one is a policy decision taken at the level of the rule rather than the case. Once it is written into a locked sheet it applies to the next request as it applied to the last, including the ones nobody has thought of yet.
Three decisions that make it an instrument rather than an opinion form
Service delivery improvement was banded in staff hours rather than in adjectives, which forces an agency to estimate rather than assert.
Administrative support was an explicit ladder of sponsorship level — taking the thing everybody already knew mattered and making it a visible, weighted factor instead of an invisible one.
The sheet was locked. Cell protection and data validation behind a password, so an agency could enter a score on each line and nothing else, with the criteria, the weights and the formulas fixed. Without that lock, every submittal becomes a negotiation about the weights, and the instrument stops measuring anything.
One more, asked for during the build and not cosmetic: the scale at the foot of the sheet slides with the total and returns a letter. A number out of 102 invites an argument about a point here or there. A grade puts a submittal in a band, and bands are what a funding conversation can actually work with.
It survived a change of Chief Information Officer
In operational use within three months, with the ranking sheet being requested by project item and identifier by that June. It became the way the City determined technology capital projects rather than one person’s method.
Under the next Chief Information Officer the same work reappears as the annual project request workbook driving capital funding prioritization, with agency requests carried into categories and priorities and capital subfunds.
An administration keeps a process it inherited only when that process is doing work nobody wants to go back to doing by argument.
The same method, four more times
It outgrew annual submittals and was rebuilt for four different questions.
An application security risk matrix, in 2012, scored twelve enterprise systems on two axes: importance to the organization — averaging legal importance, public safety, business unit importance and public use — against security risk. Raw scan alerts are not comparable between systems, so they were weighted, high counting ten, medium five, low one, then banded onto the same zero-to-four scale everything else used. That banding is what lets a vulnerability count sit in the same quadrant as a judgment about public safety.
A manufacturer matrix, run twice, ten years apart. The first round decided the 2009 security platform selection: revision 2 on 5 June 2009, “Final” on the 8th, the PDF package out on the 8th and the 10th, issued alongside the Gartner Magic Quadrant for enterprise network firewalls. The second was the 2019 firewall selection — revision 1 on 3 January under the subject Decision Matrix Quadrant for Firewall, revision 2 the next day, revision 4 on 22 January as Updated Matrix for Firewall Selection, circulated by colleagues through 4 February, with final scoring back on 5 April. Five vendor tabs read one shared sheet of criteria and weights, so changing a single weight re-scored all five together and no two candidates could be judged by different rules.
An enterprise resource planning module matrix in 2013 turned the same quadrant on fourteen modules, scoring module risk on product lifecycle, impact on the wider platform, dependencies and implementation impediment. Last saved on 18 March 2013 by the enterprise architect working it alongside me, and emailed twice on the evening of 11 April with the priority matrix following just after midnight.
And by 2014 a full procurement instrument — the same workbook again: 185 requirements across four separate instruments and 505 available points, with three weight tiers each carrying defined partial credit, every requirement graded, and every requirement group reporting its own percentage so that a weakness showed at section level instead of disappearing into a total. The citywide network study of 2023 is the last descendant on the record, still running two matrices and twenty-five criteria at weights of one to four.
Six years apart, the same four rules: declare the weights before scoring, define what each score means, keep the mechanism fixed, and resolve to one comparable figure. Everything else about those five instruments is different, and none of it matters as much.
One file, six decisions, fourteen years
The lineage above is not an argument about resemblance. It is what the files say.
Three surviving workbooks — the 2012 application security matrix, the 2013 enterprise resource planning matrix and the 2019 firewall matrix — carry the same internal creation timestamp: created by michaely, 18 March 2009 at 15:05:37. They are not similar instruments built to a shared pattern. They are one workbook, saved forward. The 2009 and 2014 rounds and the 2023 decision are the same file again.
That file was created three months before it decided the June 2009 security platform selection. It scored twelve enterprise systems for security risk in 2012. It ranked fourteen modules of an enterprise resource planning programme in 2013. By 2014 it had been built out into a full procurement instrument — 185 requirements across four separate instruments and 505 available points. It chose a firewall in 2019: its internal modified stamp reads 22 January 2019 at 18:31, matching the email that carried it to the minute. And on 17 May 2023 it was pulled out again for the web filter decision. Same file. New question.
Fourteen years and two months from creation to last recorded use, across six decisions in five different domains, under four Chief Information Officers.
The thing worth taking from that is not the longevity. It is that the method and the mechanism turned out to be the same object. The four rules below were not written down somewhere and re-applied by hand each time; they were in the sheet, in its locked cells and its shared criteria tab and its normalized axes, which is why the next person facing a different question could reach for it rather than rebuild it. An architecture principle survives as long as somebody remembers it. An architecture principle embedded in a working artifact survives as long as the file opens.
How it actually spread
Not in a straight line, and the shape is worth recording because it is the normal life of a governance instrument rather than the one people expect.
Adoption was slow at first. It rose sharply across the first two years, then fell back — and then generalized, until the shape was being reached for on more or less any decision that had to be defended rather than only on capital requests. Use peaked around 2014, eased, and the file stayed in circulation to 2023.
The dip after the first rush is the honest part. A scoring sheet is not adopted because somebody mandates it; it is adopted when a person has a decision they cannot defend and remembers that the form exists. Once the novelty wears off, the only decisions that keep using it are the genuinely contested ones — which is the population it was built for in the first place. An instrument that is used for everything is being used as paperwork. An instrument that keeps being pulled out for the hard calls, sixteen years on, is being used as an instrument.
Two notes on the dates. The span runs from the method’s authorship in March 2008 to the last use anybody can point at, 17 May 2023 — the record stops where the evidence stops, not where the use probably did. And every surface and bar on this site measures what was live in a year, weighted by what failure would have cost, rather than how hard a thing was leaned on, so this entry draws an even ridge where the paragraph above describes a curve. The span is the claim; the shape is here, where it can be read and disagreed with.
Outcome
A funded list the division could defend line by line — and one file, created 18 March 2009, still making decisions in 2023.