Scoring Methodology

Rubric version 2026.07-v0. Last reviewed August 9, 2026.

1. What this document is

This is a description of how Ariadni produces a score. It is published so that a client, a lawyer being scored, or anyone else can see what the assessment rests on and judge whether it is fair. Ariadni committed to a documented and inspectable methodology in its Conflicts and Data-Firewall Policy, and this document is that commitment.

It describes the process completely. It does not publish the contents of the evidence ledger, and section 10 says plainly what is withheld and why.

Ariadni LLC is a technology company, not a law firm. Nothing here and nothing Ariadni produces is legal advice, and no attorney-client relationship arises from using it. An assessment is the client's own private instrument. It is not Ariadni's statement, or its owner's statement, about any lawyer.

2. What Ariadni scores

Ariadni reports four things:

Responsiveness. How quickly counsel replies and turns work around, computed from the timing of mail in the client's own account.

Who did the work. What the document's own metadata indicates about who drafted and edited it, compared against the lawyer the client was told was doing the work.

Work quality. Observations about the document itself, anchored to specific text in it.

Value. Whether the fee or rate the client is paying is consistent with what that document reflects.

The four are reported separately and also combined into one overall figure. The client controls how the four are weighted, globally or per matter, and can change the weighting at any time and see the overall figure move. The model that reads the document is never told the weights, so the weighting is applied to the observations rather than influencing them.

3. What Ariadni reads, and what it keeps

Monitoring is forward-only, from the moment a client connects an account. There is no sweep of mail history.

Ariadni reads the mail the client designates, from the counsel domains the client lists. Its access cannot send anything on the client's behalf; it can create drafts the client reviews. Documents are read for a single analysis and are not retained. No Ariadni server stores client documents or mail.

Before any document text reaches the analysis model, it is de-identified on the server. That de-identification is best effort and is not a guarantee. The only content sent for analysis is such text, and during the pilot it goes to Anthropic under Ariadni's own metered account. Names are restored locally afterward.

What persists is the derived result rather than the source: the running scores, the de-identified assessment history, the signed score records described in section 14, the client's profile and matter setup, and a status log. Every account is self-scoped. There is no administrative path and no cross-account path through the product by which one client's material could be read by another, or by Ariadni.

4. How the rubric is built

The rubric is not a fixed opinion held in code. It is compiled from an evidence ledger.

The ledger holds one record per rubric item. Each record states a single checkable proposition, the document types and jurisdiction it applies to, the class of signal it belongs to, its provenance, and its effect on scoring. A deterministic compiler turns the qualifying records into the instructions the analysis runs on, and produces a hash of the result.

Two consequences follow, and they are the point of the design. The same ledger always compiles to the same rubric, so a score can be reproduced. And a disputed score can be traced back to the exact set of items that produced it.

5. Classes of evidence, and what each may do

Not all evidence is equal, so the ledger does not treat it as though it were. Every item carries a class, and the class determines what the item is allowed to do.

Active. Affects a score. Cited evidence in force.

Legacy. Affects a score. Carried over from the first version of the rubric so that its behaviour was preserved exactly when the ledger was introduced. Each one is a citation owed, and each is being backfilled.

Candidate. Does not affect a score. Proposed, under research, not in use.

Observed. Does not affect a score. Measured from data rather than drawn from authority.

Retired. Does not affect a score. Withdrawn or superseded, kept so the reasoning stays on the record.

Only active and legacy items are compiled into the rubric. Candidate, observed, and retired items are never rendered and cannot reach a score. This is enforced mechanically and tested on every change, not left to care.

The observed class deserves a specific note, because it is the one that could be abused. It exists for signals measured from data, such as how often a provision appears. Measurements of that kind cannot be checked by a third party against a source of authority, so they are held to a lower authority rather than admitted as though they were citations. An observed item may raise a question. It may never, on its own, lower a grade. It may prompt research into an item. It may never be the citation that justifies one. As of the version stated at the top of this page, no observed item exists.

6. What it takes for evidence to enter

For an item to become active and affect scoring, it must carry a citation with a pinpoint and a verbatim quote that a third party can look up and check. A general reference to a source is not enough. An assertion that something is standard practice is not enough.

For an item to be admitted as observed, it must clear a floor on the number of distinct firms, matters, and clients it was measured across, so that one firm's drafting habit cannot be reported as a market fact. It must state what population it was measured on and, explicitly, what population it was not measured on. It must record the schema and the window that produced it.

Those floors may be raised. They cannot be lowered without a recorded decision, because lowering them is precisely what would turn the class into the thing it exists to prevent.

7. The lawyer-agnostic test

The controlling governance test is that the rubric must score any lawyer's work the same way, including work by the firm founded by Ariadni's owner. Three rules give that effect.

No ledger item may name or characterise a particular lawyer, firm, or school of drafting. The only permitted stratification is by anonymised rate band. A specific lawyer or firm is never named in a comparison.

Ariadni's owner holds a veto and not authorship. He may reject an item as inaccurate, out of scope, or wrongly cited. He may not add an item because he believes it is correct. Every veto is recorded.

There is one pipeline. No firm is special-cased, in either direction, and there is no path in the product by which a score can be edited after it is produced.

The first of these is checked mechanically on every change, alongside the other automatic checks in section 9.

8. Versions, and why a past score is never restated

The ledger changes as evidence is added and corrected. The compiled rubric is cut as a numbered version monthly, so there is a stable artifact to audit against.

Every score records the rubric version and hash that produced it. A matter is scored on the version that was in force when it began. When a newer version exists the client is told, and may re-run the portfolio against it and see both. A past score is never silently restated. If a rubric item is later found to be wrong, the honest record is that the score was produced under the version then in force, not a quietly corrected number.

9. What is checked automatically before any change ships

A change to the rubric cannot ship unless a set of assertions passes. They do not check whether a score is correct, which is a matter of judgment. They check the things that are true or false without an opinion:

The compiler is deterministic, so the same ledger produces the same rubric.

Only active and legacy items are rendered. Candidate, observed, and retired items are confirmed absent from the compiled output.

Every active item carries a full citation, and no active item rests on a source too weak to be the citation of record.

No item names a lawyer, a firm, or a first-person preference.

A retired item is documented with the date and reason it was withdrawn, and carries no citation that could be read as though it were still in force.

An observed item clears every floor and characterises its population.

The structure of the report the client receives is intact.

Separately, the de-identification is measured against planted identifiers on a fixed set of documents, and a change that leaks one does not ship.

10. What this methodology does not publish

This page describes the process. It does not publish the contents of the evidence ledger: the individual items, the sources behind each one, the internal numeric mappings, or the text of the analysis instructions.

The reason is straightforward and is stated rather than hidden. That material is the product. Publishing it would let it be copied without contributing anything to the fairness of an assessment, because what makes an assessment fair is the standard evidence must meet, the classes that limit what evidence may do, the neutrality rules, and the version discipline. All of those are published above.

The ledger is available for inspection in circumstances where inspection is warranted, including a good-faith dispute about a specific score, under an appropriate confidentiality arrangement. Withheld is not the same as unreviewable.

11. Limits of a score

A score is an observation about a document and an engagement. It is not a verdict on a lawyer.

The analysis produces observations and questions, never advice, recommendations, or instructions to act.

Every observation is anchored to specific text. The analysis does not make conclusory judgments about a lawyer's character, competence, or reputation. It assesses the work, not the person.

There is no outcome data behind any of it. Nothing here supports a claim that an absent provision would have caused harm, only that it is absent.

The analysis sees what it is given. A clean draft without the counterparty's version cannot show what the lawyer changed, and the report says so where that applies.

Document metadata is evidence and not proof. A mismatch between metadata and the lawyer of record is a reason to ask a question, not a finding.

De-identification is best effort and is not a guarantee.

A lawyer may have a good reason for every choice the analysis notices. The output is framed so a client can ask rather than accuse.

12. How to question a score

A client, or a lawyer whose work a client has had assessed, may raise a specific score. A challenge that identifies the document and the observation will be answered with the rubric version and hash that produced it and the items in force at that time.

Where an item is shown to be wrong, it is corrected in the ledger and the correction takes effect in the next version, on the record. Scores already produced are not silently changed, for the reason given in section 8.

13. Version and review

This page describes rubric version 2026.07-v0 and was last reviewed on August 9, 2026. It is reviewed when a rubric version is cut. If the version stated at the top of this page is behind the version shown on a recent report, this page is the one that is out of date, and the discrepancy should be reported.