benjivers/jev-mcp Public Public: anyone can read this; nobody can change it.

workbenchthis machine
Files 16904894ed3e75ecfac04ccf7b61044217125dad
243 lines · 10.2 KB · ~2.6k tok Raw History
1
# jev-mcp
2
3
An MCP server that exposes [TypeSafe Jev](https://docs.typesafe.ai) as typed judgment tools.
4
5
Jev is a System One model. It returns a typed answer and a calibrated probability
6
distribution, never prose. These tools surface that faithfully rather than hiding
7
it behind a label.
8
9
## Install
10
11
Requires Node 20.12+ and a TypeSafe API key from
12
[console.typesafe.ai](https://console.typesafe.ai/settings/keys).
13
14
### Claude Code plugin
15
16
One marketplace add, one install. This registers the server and the agent skill
17
together, so there is no separate skill step.
18
19
```bash
20
/plugin marketplace add rashedInt32/jev-mcp
21
/plugin install jev@jev-mcp
22
```
23
24
The plugin runs the published package through `npx -y jev-mcp@<version>`, so nothing
25
needs building. The first launch downloads the package, so allow a few seconds before
26
the server shows as connected in `claude mcp list`.
27
28
### Any MCP client
29
30
```bash
31
claude mcp add --scope user jev -- npx -y jev-mcp
32
```
33
34
Or in a client's JSON config:
35
36
```json
37
{
38
  "mcpServers": {
39
    "jev": { "command": "npx", "args": ["-y", "jev-mcp"] }
40
  }
41
}
42
```
43
44
Do not register the server directly **and** install the plugin. Two servers named
45
`jev` will otherwise both register.
46
47
### The key
48
49
The server reads the key from `TYPESAFE_API_KEY` in its environment, then from
50
`JEV_API_KEY`, then from `~/.config/typesafe/key`. It is never a tool argument, so it
51
cannot land in a transcript or in a model's context.
52
53
The key file is the most reliable source, because some MCP clients strip the
54
environment before spawning servers:
55
56
```sh
57
mkdir -p ~/.config/typesafe
58
printf '%s' "ts_..." > ~/.config/typesafe/key
59
chmod 600 ~/.config/typesafe/key
60
```
61
62
If you prefer the variable, put it in `~/.zshenv` rather than in any repo, and make
63
sure it is **exported**. Without `export` the variable exists only in the shell that
64
read it, and every server Claude Code spawns fails with a missing-key error.
65
66
### Plugin internals
67
68
The server is declared **inline** under `mcpServers` in `.claude-plugin/plugin.json`.
69
There is no `.mcp.json` anywhere in the repo, and that is deliberate.
70
71
A `.mcp.json` at the plugin root is auto-discovered by the plugin loader, so it works.
72
But any session opened in this directory *also* reads that same file as a project
73
config, where `${CLAUDE_PLUGIN_ROOT}` is undefined. The result is a missing-variable
74
warning and a scope conflict on the same server name. The inline form has exactly one
75
loader and produces neither. Verify with `claude mcp list`: the server appears as
76
`plugin:jev:jev` and the diagnostics section stays empty.
77
78
One trap. `claude plugin details jev` reports `MCP servers (0)` for an inline
79
declaration even while the server is connected and working. That is a gap in the
80
inventory count, not a failure. Trust `claude mcp list` over `plugin details` here.
81
82
The plugin deliberately ships **no `env` block**. Naming the key there would expand to
83
an empty string when the variable is unset, and an empty string is not nullish, so it
84
would shadow the `~/.config/typesafe/key` fallback and turn a working setup into a
85
missing-key error. Leaving `env` out keeps all three key sources live.
86
87
## Tools
88
89
| Tool | Primitive | Use when |
90
| --- | --- | --- |
91
| `jev_classify` | Choice | The answer is one of a fixed set you define |
92
| `jev_score` | Score | The answer is a degree on an ordered scale |
93
| `jev_check` | Noul | The answer is yes or no, and you want the probability |
94
| `jev_ask` | all three | You have several questions about the same state |
95
| `jev_models` | — | Confirm the key works and find a model id |
96
97
Every tool returns the full probability distribution alongside the answer, plus
98
`confidence` for Choice and Score. Results come back as MCP structured content, so a
99
client gets typed data rather than a JSON string to re-parse.
100
101
### Prefer `jev_ask`
102
103
Jev prefills the state once and scores every question in a single forward pass, so
104
extra questions add almost no latency or cost. TypeSafe's own
105
[measurement](https://docs.typesafe.ai/cookbooks/parallel_questions) on a
106
document-dominated workload puts one batched call at 12.2x cheaper and 10.0x faster
107
than one call per question, with no change in the answers.
108
109
Questions in one request cannot see each other's answers. State any speculative
110
premise explicitly and let your own code decide which answers apply.
111
112
## Design rules
113
114
1. **The caller owns the option set.** `jev_classify` requires options from you, so
115
   the model can pick the wrong one but can never invent one. A selector cannot
116
   choose a candidate the enumerator dropped. This is the single most common way
117
   these integrations fail.
118
2. **Probabilities are always returned.** Not just the winner.
119
3. **The key lives in the environment.** Never in an argument.
120
4. **Nothing is silently dropped, overwritten, or truncated.** A request that cannot
121
   be honoured exactly fails with a reason instead of quietly changing meaning.
122
5. **Only JSON-RPC reaches stdout.** Logs go to stderr, always.
123
124
### The no-match option
125
126
Each selecting tool adds a `none` option by default so the model can decline rather
127
than being forced to pick. Turn it off with `add_none: false` when one option must
128
always apply.
129
130
If you already use the name `none` for an option of your own, the added option takes
131
a different key instead of overwriting yours, and the response reports which key
132
carries the no-match meaning in `none_option`. Your option and its probability always
133
survive intact.
134
135
### Confidence gating
136
137
Choice and Score answers include an `action` of `act`, `review`, or `abstain`, derived
138
from `confidence` and the thresholds you pass in `act_above` and `review_above`
139
(defaults 0.8 and 0.5). `jev_check` returns a `verdict` of `yes`, `no`, or `uncertain`
140
from `yes_at_or_above` and `no_at_or_below` (defaults 0.7 and 0.3).
141
142
These defaults are starting points, not universal rules. Calibrate them on your own
143
data and on what it costs to be wrong; a destructive action deserves a higher bar than
144
a read-only one. Confidence describes how concentrated the distribution is. It is not
145
a claim that the answer is correct.
146
147
A Noul near 0.5 means yes and no are close to equally likely, not that the answer is
148
"medium", which is why the middle band reports `uncertain` rather than rounding.
149
150
### Errors
151
152
Failures come back with `isError` and a classified body: `kind`, `retryable`, and
153
where available `status`, `requestId`, and a `hint`. A rejected key (`authentication`,
154
never retryable) is distinguishable from a rate limit (`rate_limit`, retryable) and
155
from a malformed question (`invalid_request`). The SDK already retries 408, 429, and
156
5xx with backoff before an error surfaces here.
157
158
### Limits
159
160
A Choice question accepts at most 255 options, which is an API limit; past that,
161
search in two passes, one question picking a window and a second ranking within it.
162
Question count and state size are local caps that bound cost on a single call, and
163
both are configurable. Oversized state is rejected rather than truncated, because
164
truncating silently changes the material the judgment rests on.
165
166
## Agent skill
167
168
`skills/jev/SKILL.md` teaches an agent when to reach for these tools and how to shape
169
the call. It is a bridge, not a tutorial. Primitive semantics, state design, and
170
composition patterns live in TypeSafe's own `typesafe-ai` skill and in the docs, so
171
this one deliberately does not repeat them.
172
173
Its first section is a three-way test: answer it yourself, call a tool, or write SDK
174
code. That test follows the same line as [When not to reach for this](#when-not-to-reach-for-this)
175
below, so an agent loading the skill does not end up arguing with this README.
176
177
Point your client at the directory, or copy the file to `~/.claude/skills/jev/`.
178
179
## Configuration
180
181
| Variable | Effect |
182
| --- | --- |
183
| `TYPESAFE_API_KEY` | Required. `JEV_API_KEY` also works. |
184
| `JEV_MODEL` | Model id. Defaults to `jev-latest`. |
185
| `JEV_TIMEOUT_MS` | Per-attempt timeout. Defaults to 15000. |
186
| `JEV_MAX_QUESTIONS` | Questions per `jev_ask`. Defaults to 64. |
187
| `JEV_MAX_STATE_CHARS` | Largest state accepted. Defaults to 200000. |
188
| `TYPESAFE_LOG_LEVEL` | SDK verbosity. Safe at any level; all output goes to stderr. |
189
190
An unusable value for any numeric setting falls back to the default and warns on
191
stderr, rather than becoming `NaN` and disabling the limit it was meant to enforce.
192
193
## Troubleshooting
194
195
**Every call reports a missing key.** Some MCP clients filter the environment before
196
spawning servers, which drops `TYPESAFE_API_KEY`. Confirm the variable is exported,
197
then pass it explicitly in the client's server config if it still does not arrive.
198
Run `jev_models` to check the key in isolation.
199
200
**The server connects and then dies.** On a stdio transport, anything written to
201
stdout that is not JSON-RPC breaks the connection. This server routes all logging to
202
stderr, so `TYPESAFE_LOG_LEVEL=debug` is safe to turn on while debugging.
203
204
## When not to reach for this
205
206
Jev earns its place on a decision that repeats thousands of times inside software,
207
where code can enumerate the options first and you need a number to threshold on.
208
209
For a one-off judgment during a conversation, an ordinary model answer is usually
210
better, because it comes with reasoning you can argue with. Jev gives you a number
211
and a label. That is a feature at scale and a limitation in dialogue.
212
213
If deterministic code already decides the case correctly, keep the deterministic
214
code. Typed output guarantees the interface, not the truth. Measure before adopting.
215
216
## Development
217
218
```bash
219
npm ci
220
npm run build
221
npm run typecheck
222
npm test          # offline: unit tests plus regression tests against a local stand-in
223
npm run test:e2e  # live, requires TYPESAFE_API_KEY
224
```
225
226
To run your local build instead of the npm release, register it directly and skip
227
the plugin, since the plugin always launches the published version:
228
229
```bash
230
claude mcp add --scope user jev -- node /absolute/path/to/jev-mcp/dist/index.js
231
```
232
233
Releasing: bump `version` in `package.json`, `.claude-plugin/plugin.json`, and
234
`.claude-plugin/marketplace.json` together, then `npm publish`. The plugin cache is
235
keyed by version, so an unbumped plugin keeps serving the old snapshot.
236
237
The offline suite runs the real built server over stdio against a local stand-in for
238
the TypeSafe API and asserts on the request bodies it actually sends, so a regression
239
in what reaches the model fails the build.
240
241
## License
242
243
MIT
↑↓ navigate ↵ open esc close
↑↓ navigate ↵ open esc close

Keyboard shortcuts