JevAct (einptein, jev1-2b-v2)

Availability and evidence

Openness

Marked closed in the published row

License note

not stated (the author published only an endpoint and the evaluation client)

Cost evidence

estimated; the board’s row disclosure contains the published basis.

Endpoint condition

author-hosted endpoint (one machine at 115.190.124.200:18084, China), reached from a Hetzner server in Germany

Note on this row

Requested in GitHub issue #66. The author published an endpoint and an evaluation client, not weights, and his API is not the TypeSafe wire format, so JevBench's own `jevact_api` adapter implements exactly the mapping table in his eval_code.zip: state to state, instructions to the question, the labels with their criteria text as the options in label order, noul as false/true, and results[0].options[i].probability read back as the probability of labels[i] by index. His server answers HTTP 400 with `all inference items exceeded max_context_tokens or contained reserved markers` for an input it cannot take, and his own client and test script record exactly that as one failed item and continue; the adapter therefore surfaces that one documented refusal as the harness's 422 refusal path, which scores it as a wrong answer and does not count toward the stop rule - the same treatment the swanOne and Cygnet packages get for their own 422. Any other 400 or an outage still stops the run. The endpoint reports no token accounting, so cost is the labelled 2B size-class estimate. The endpoint is one machine in China and the latency includes that distance; it is not a production API and gets the standard x2 self-host adjustment, without the +0.15 s that only our own servers carry. 237/308 sealed items answered validly (failures count as wrong)

Published source

https://github.com/fstandhartinger/jevbench/issues/66

From the public v1.4.2.2 aggregate. Scores and ranks can change when a new release is published.