GuidesRuntime
Kiso, Hinoki's Rust server¶
Kiso is the HTTP server inside Hinoki's native extension, written in Rust on
hyper, tokio and rustls. It is what hinoki dev and hinoki serve run unless
you choose Granian, and it can answer many requests without entering Python.
hinoki serve --workers 4
# notes/settings.py
from hinoki import settings
class ProjectSettings(settings.AppSettings, prefix="NOTES_", dotenv=".env"):
server: settings.ServerSettings = settings.ServerSettings(server="kiso", workers=4)
--server kiso selects it explicitly when a project defaults to Granian. Choose
Granian instead when you need an ASGI wrapper around the application
(Serve an application).
What is native and what still runs in Python¶
The listener, HTTP/1.1 parsing, connection and header limits, body framing and its limits and timeouts, route matching, the request ID, and the answers that need no application code run on the native runtime without taking the Python interpreter lock. Without host, CORS or HTTPS-redirect policies, a request the native layer can answer by itself is answered there: unknown paths and methods (404, 405 with Allow), oversized request lines or headers, 503 while the worker is not ready or the in-flight limit is reached, and malformed requests. When one of those policies is active, the Rust classifier checks it first and then answers unmatched requests itself, with the same CORS headers as the Python pipeline. The response carries the same security headers, request ID, CORS and HSTS policy as one produced by the application.
The lifespan, handwritten handlers and their database calls still use Python. The OpenAPI document and the documentation pages and assets are built once at startup by the same Python code and answered by the Rust classifier for GET and HEAD (other methods continue to route lookup), with the same status, headers, request ID, CORS and security headers; they cross into Python only when a request target is not plain ASCII. Host allow-list checks, CORS preflight answers and HTTPS redirects run in the Rust classifier before docs and route lookup, with the same response order and headers as the Python pipeline. For matched routes that continue through the Python pipeline, Kiso also percent-decodes the captured path values and passes them to Python; an invalid UTF-8 escape is carried as an error marker and reported at the same point where the pipeline previously decoded it. Python still applies str, int, UUID, bool and custom converters, and owns authentication, hooks, body/query binding, handlers, error translation and response assembly.
For an eligible public GET, DELETE, POST, PUT, PATCH or QUERY function, Kiso skips Python route matching and request binding. It accepts parameters from the path and query string annotated str, int, float, bool or UUID with the built-in converter, and query parameters whose default has exactly that type. Rust decodes provided values, converts them with the same rules as the Python converters (an invalid value gives the same 422), validates the full query, negotiates Accept, and prepares the handler arguments. An absent optional query uses its declared default, including when a wrapper observes the keyword arguments; an empty value is still an explicitly supplied empty string. Missing required or repeated query parameters use the same validation response as the Python pipeline. Error translation and Response creation remain in Python; JSON encoding uses Hinoki's Rust encoder, and Rust finalizes the response headers and suppresses the body for HEAD. Those final headers include the CORS headers for the request's Origin, on successful and error answers alike, exactly as the Python pipeline adds them; an answer that sets its own Access-Control-Allow-Origin keeps its CORS headers. An async handler runs in its ordinary asyncio task, unless it qualifies for the native handler lane; a sync handler uses the bounded executor. If a sync handler returns an awaitable, Hinoki awaits it on the same request task after the executor call completes. An integer longer than 18 digits is left to the Python pipeline, which applies Python's own integer limits. A body method is eligible with exactly one Schema body parameter and only built-in parsers (JSONParser, FormParser, TextParser, BytesParser, exact types without overrides; a custom parser or a JSONParser.prepare override uses the Python pipeline). The route's parser is chosen from the Content-Type after the body is read, so an unlisted type is the same 415. Its body is read from the native connection only when the handler's turn comes, after content negotiation, with the same limits, and is then decoded and validated by the Rust decoder and validator. The failures keep the order and the answers of the Python pipeline: a declared length above the limit or a body read failure first, then a malformed query string, then the content type (415) and malformed JSON (400), then one 422 listing every path, query and body problem in declaration order. A refusal before the body is read closes the connection, as on the Python pipeline. The automatic path does not accept several body parameters, file or Request parameters, custom parsers, multipart uploads, defaults on path parameters, custom response headers or custom renderers. The built-in JSONRenderer, HTMLRenderer, TextRenderer and XMLRenderer, alone or several on one route, are eligible: Rust negotiates Accept (including 406) and Python renders with the chosen one, exactly as on the Python pipeline. A subclass of a built-in renderer or one with extra attributes is custom. A route with an access policy stays eligible, with at most one Caller principal parameter: Python authenticates the request before anything else, so the order of the answers (400, invalid_query, 401/403) matches the Python pipeline, and the body is still read only after authentication succeeds. When that access policy uses an unmodified ClaimsJWTAuth, Rust also answers the refusals itself (401 with the challenge, or 403) without crossing into Python, and hands the admitted caller to the handler. A route with a rate_limit policy is eligible too: Python asks the policy, with the same client address as the Python pipeline (the forwarded client when the TCP peer is a trusted proxy, otherwise the peer) and the authenticated caller, right after authentication and before negotiation, the body and validation, so a refused client gets the same 429 or 503 without the server reading its body. A route with atomic=True stays eligible: the handler runs inside the same transaction as on the Python pipeline, the commit follows the encoding of the answer, and any error or timeout rolls back before it is translated. Host restrictions, CORS and HTTPS redirects are checked natively and do not make an otherwise eligible route fall back. Every other route uses the full Python request pipeline for binding, validation, error translation and response assembly. The request body is read natively only when that pipeline asks for it, after authentication, so an unauthenticated client cannot make the server buffer a body. Handlers themselves remain Python code; this is not an application written in Rust, and no performance figure is claimed here. A Query[Schema] parameter of a GET or DELETE is eligible too (Python converts and validates the query string, reporting errors in the pipeline order; it does not combine with a body). A Form[Schema] or XML[Schema] body is eligible too: the text is converted and validated as described in Form bodies and XML bodies.
The native handler lane¶
An async def handler that only talks to the database through the ORM can run without an asyncio task: Kiso drives its coroutine itself, runs each database wait on the native runtime and sends the answer from there. Hinoki decides this once per route at worker startup, by reading the handler's own source. A route takes the lane when:
- it is a
GET,DELETE,POST,PUT,PATCHorQUERYthat is eligible for the native path above (a body method takes oneSchemabody and built-in parsers); - the function is a plain
async def, not wrapped by a decorator; - every
awaitin it is an ORM call on aModelclass:- a read:
Model.aget(...),Model.aget_or_none(...), orModel.filter(...)/Model.all()withfilter,order_by,limitandoffset, ending inaget,afirst,acountoraexists, or awaited directly; - a write:
acreate,aupdate,aupdate_where,adelete,abulk_create,abulk_update,abulk_delete, oraupdate/adeleteending afilter(...)chain. A write counts only on a model that overrides none of the write hooks (before_save,before_create,after_create,after_save) or the methods the write calls (insert,asave,from_schemaand the like);asave,aupdate_fromandaupdate_withinon an instance stay on asyncio.
- a read:
These keep a route in an asyncio task: async with, async for, yield, async comprehensions, local names that shadow a model, and any other await (asyncio.sleep, an HTTP client, another coroutine).
- Guards and bodies. A guarded route (
access,rate_limit,Query[Schema]or several renderers) is served the same way, and so is a body method: its guards run first inside the driven coroutine, in the order of the asyncio path (credentials, then the rate limit, then the body and the request values), and the body is read on the native runtime. - Class APIs. A handler of a class API takes the lane when the class keeps the stock constructor (no
__init__or__new__of its own in the class or its bases) and adds noresponse_headers. The instance is built per request, as on the asyncio path; a class that builds itself with application code stays on asyncio. The constructor is read once at startup. - Transactions. An
atomic=Trueroute runs its handler inside a transaction opened, committed or rolled back on the native runtime, and an error is translated only after the rollback, as on the asyncio path. - Invalid requests. A request Rust already finds invalid (a bad path encoding, an invalid path or query parameter, an unacceptable
Accept) stays on the asyncio path. - Moving to asyncio. A provider or policy that awaits something only asyncio can run (a custom provider calling an HTTP service, a rate limit store on the thread pool) moves the request to an asyncio task at that point, once, with what is left of the timeout. A transaction already open goes with it, so later savepoints still nest and the rollback is unchanged. The same happens if a write hook is added to a model after startup and awaits asyncio.
hinoki routes --lanes shows the choice, and hinoki.runtime logs the reason for every route at debug level.
The behaviour is the same on both paths: status, headers, body, error translation, the request ID and the request_timeout (a 504). An error from a driven handler, and a failure of a generated CRUD action, is translated in place when no error handler you registered can match it; an exception one of your handlers matches is translated in an asyncio task, because the handler may await. Each request has its own context, so request_id and the database are the ones of that request. The handler's steps after its first run on the event-loop thread, so a task it starts still finds the loop. If a driven handler awaits something only asyncio can run (a database backend without native waits), it is moved to an asyncio task at that point and continues with what is left of the request timeout. At shutdown, running handlers are cancelled like request tasks.
What it assumes: model classes and their ORM methods are not replaced after startup. Set server.native_dispatch = false to keep every handler in an asyncio task. No performance figure is claimed here.
hinoki routes --lanes shows, per route, whether Kiso runs it natively and, if
not, why (Inspect an application).
Generated CRUD¶
Generated CRUD plans can also execute public create (POST), update (PATCH) and delete actions in Rust. These routes use the native path only when their model and schemas are supported and they have no assignment rule or custom CRUD hook (an owner is allowed only with identity_key = caller_id, a text non-key column outside the update schema, on any supported database), and their access policy, if any, uses an unmodified ClaimsJWTAuth. Rust reads and validates the bounded request body, runs the database transaction, fetches the written row where the response requires it, and encodes the response before commit. Writes continue to completion if the client disconnects. A unique-constraint violation returns 409 Conflict by default; a registered application error handler takes precedence. Routes that are ineligible or become stale use the ordinary Python CRUD pipeline. Plans are selected automatically at worker startup and checked once per second; a changed plan is disarmed until the worker restarts. This path adds no public setting.
The measured phase 1 path is documented in the native CRUD orientation run. It is a single-host, workload-specific measurement and does not establish general or production performance.
Limits¶
- HTTP/1.1, plus HTTP/2 over TLS when
server.http2is on (see below). No WebSocket or other protocol upgrades: anUpgradeheader is ignored and the request is answered as plain HTTP, and the server never answers101. There is no cleartext HTTP/2 (neither prior knowledge norUpgrade: h2c). - A request line plus headers larger than
execution.max_header_bytes, or a query longer thanexecution.max_query_bytes, is refused (431,414). A body larger thanexecution.max_body_bytesis refused with413, byContent-Lengthbefore any byte is read or, for chunked bodies, as soon as the total passes the limit. A client that sends nothing forbody_read_timeoutseconds (10 by default, never more than half ofexecution.request_timeout) gets408, and so does one that trickles data: the whole body must arrive within six timesbody_read_timeout. - A client that does not finish its request headers within
header_read_timeoutseconds (10 by default) is disconnected; the same timer closes idle keep-alive connections. - TLS is optional: set
server.tls_cert_fileandserver.tls_key_file(PEM; the key may be PKCS#8, PKCS#1 or SEC1) and Kiso speaks HTTPS with rustls (TLS 1.2 and 1.3, the library's default cipher suites, no client certificates, ALPNhttp/1.1, orh2thenhttp/1.1withserver.http2). The files are read once at start, so a renewed certificate needs a restart (a rolling restart of the workers is enough). A plaintext client on a TLS port gets a TLS alert, never an answer, and the handshake must finish withinheader_read_timeout. Requests over TLS have the schemehttps, so the HSTS policy applies. Granian takes the same two settings (the framework asks it for TLS 1.2 and up and HTTP/1.1 only, as Kiso does, unlessserver.http2is on). A trusted proxy'sX-Forwarded-Protooverrides the scheme,httpincluded, on both servers. The handshake and the request head each have their ownheader_read_timeout, so a slow client can hold a connection slot for up to twice that time. Behind a proxy that terminates TLS, leave them unset and list the proxy intrusted_proxiesinstead. - HTTP/2 is opt-in: set
server.http2 = truetogether with the TLS files (it is a settings error without them, on both servers). The server then offers ALPNh2andhttp/1.1and serves each connection with the protocol it negotiated; a client that offers onlyhttp/1.1, or no ALPN, still gets HTTP/1.1. Requests reach the application unchanged, withhttp_version2in the ASGI scope; the:authoritypseudo-header becomes theHostand wins over ahostheader that differs. Limits are fixed, not settings: 32 concurrent streams per connection (the 33rd is refused), a 64 KiB stream window, header lists bounded byexecution.max_header_bytesand at most 100 header lines per request (cookiefields count once; more is answered431), and the h2 library's rapid-reset protections. A connection counts as one slot ofmax_connections, but every stream in flight uses backend capacity (the in-flight limit and the queue) like a request of its own, so size those limits for connections times 32. The server also drops a connection on which more than 100 streams end before their response completed (client resets) within a fixed 10-second window; it is closed without aGOAWAY. The count is per connection, not per client: a client can pass about twice the bound across a window boundary, and a new connection starts from zero. A connection with no open stream forheader_read_timeoutis closed gracefully (GOAWAY; a stream opened in the race with theGOAWAYis accepted and then cut when the shutdown timeout ends). A response that sends no data forwrite_timeoutseconds (a client that keeps its flow-control window at zero, or stopped reading) also gets the connection aGOAWAY: hyper cannot reset one stream from outside, so the other streams have the shutdown timeout to finish and the stalled one is cut with the connection. Progress is tracked per response, so opening new streams does not hide a stalled one. Keep-alive pings every 30 seconds drop a peer that stopped answering. Setwrite_timeoutabove 32 x 16 KiB divided by the slowest client bandwidth you expect to serve: HTTP/2 shares the window between streams unevenly, so on a slow link each of 32 concurrent downloads can wait that long between two chunks and be taken for stalled while bytes still flow. A request with severalhostheaders and no:authorityis refused with400. A client that trickles the head of one stream while others are open is not bounded byheader_read_timeout. On stop the server sendsGOAWAY, lets streams in flight finish, and aborts what is left after the shutdown timeout. Not provided: cleartext HTTP/2 (h2c), server push, and WebSocket orCONNECTover HTTP/2. Granian withserver.http2negotiates HTTP/2 itself (httpmodeauto), under its own limits. - A client that stops reading a response for
write_timeoutseconds (30 by default) is disconnected; any progress restarts the timer. max_connections(10000 by default) bounds open connections. Requests waiting for the application are bounded by an in-flight limit and a queue; beyond them the answer is503.- Requests with ambiguous framing (for example
Content-Lengthtogether withTransfer-Encoding, repeated lengths, or a bare line feed in the head) are refused with400and the connection is closed. - On start, Kiso names itself on stderr before the
listening on ...line (which stays on stdout, where tools wait for it): its version, the mode (serveordev, which reloads on file changes), the worker count, the scheme, whether the access log is on and whether the native handler lane is on. In a terminal (and withNO_COLORunset) it is two lines with the name in bold; otherwise it is one plain line, for exampleKiso 0.0.1 (Hinoki's Rust server): serve, 4 workers, http, access log off, native lane on. - The access log is one line per answered request, written to stderr by Kiso:
127.0.0.1:52341 - "GET /items?limit=5 HTTP/1.1" 200 OK 3.2ms. The time is how long the server took to decide the answer, not including writing it to the client, and the line is printed for every answer, including the ones the server gives itself (404,429,503). When stderr is a terminal (andNO_COLORis unset) the line is coloured in the style of a development console:INFO:in green, the request in bold, the status by class (2xx green, 3xx cyan, 4xx yellow, 5xx red) and the time dimmed. Control characters in the target are shown as\xNN. Other formats and filters:server.access_log_format = "json"writes one JSON object per line instead, for a log pipeline:{"time":"2026-11-19T19:56:25.881Z","peer":"127.0.0.1:52341","method":"GET","path":"/items","query":"limit=5","http_version":"1.1","status":200,"duration_ms":3.2,"lane":"python"}.timeis RFC 3339 UTC.lanesays where the request was first routed:earlyfor an answer from the transport or the classifier,nativefor a generated CRUD plan that Rust executes, andpythonfor every request handed to the Python side, including handlers on the native handler lane, so it can differ from whathinoki routes --lanesshows.routeis the pattern that matched, e.g./items/{id}, so requests can be grouped without parsing paths (absent when no route matched). Quotes and control characters are escaped.server.access_log_format = "logger"hands each request to the Pythonhinoki.accesslogger instead of writing to stderr, so your own handlers, formatters and filters (see your own loggers) decide where it goes; each record carries the JSON fields asrecord.access. Kiso queues up to 4096 records and a logging thread passes them to Python in batches, taking the GIL once per batch; when the logger falls behind, new records are dropped rather than slowing requests.- Setting a format turns the log on, in
servetoo, unlessaccess_logisfalse. access_log_skip = ["/health", "/static/*"]leaves noisy paths out andaccess_log_slow_ms = 100keeps only slow answers; failures (status 400 and up) are always logged.
- A client that disconnects does not cancel its running handler; the handler finishes and its answer is discarded.
- A WSGI callable (
--interface wsgi) runs in Python threads behind this listener, not on the native lane: see WSGI on Kiso for what applies to it.
Workers, signals and restarts¶
With one worker hinoki serve runs the server in its own process. With several workers (and always under hinoki dev) it starts a supervisor that opens the listening socket once (honouring backlog), starts each worker as a fresh process that imports the target and builds the application, and passes the socket down. Workers share the one socket, so connections are spread by the operating system's accept; Hinoki does not balance them. Databases and pools open inside each worker's lifespan, never in the supervisor.
SIGTERM,SIGINT(Ctrl-C) orSIGHUPstops accepting, lets running requests finish for up tolifecycle.shutdown_timeout, runs the lifespan shutdown and exits. A worker still running five seconds after that is killed. A second signal kills the workers immediately. Once the stop begins the supervisor closes its own copy of the socket, so a client that connects while the workers drain is refused.--workersmust be 1 to 256 (0usesserver.workers).- A worker that exits on its own, whether it failed to start or crashed, is not restarted: the supervisor stops the other workers and exits with a non-zero status. Restart policy belongs to the process manager (systemd, a container runtime) that runs
hinoki serve. - The listening socket is opened before the lifespan startup of a supervised worker finishes, so connections wait in the backlog until a worker accepts them. With one worker the socket is opened after a successful startup.
- If the supervisor is killed without a chance to clean up, its workers notice within a second of finishing their startup and stop. The workers' copy of the listening socket is not inherited by processes a handler starts.
- A socket that cannot be opened (address in use, a bad port or host) is reported as
error: cannot listen on HOST:PORT: ...with exit status 2, with one worker or several.
Reload with hinoki dev¶
The supervisor watches the project directory (*.py, *.toml and .env files; .venv, .git and caches are ignored) and, on a change, stops the worker gracefully and starts a new one on the same socket, so a client connecting during the restart waits instead of being refused. If the new code fails to import or start, the error is printed (a short error: cannot load ... line for a missing module or name, a traceback for other failures), waiting for changes follows, and the next edit starts the worker again; connections wait in the meantime. The host, port, backlog and the choice of server are read once when hinoki dev starts: change them and restart the command. The other server settings (limits, trusted_proxies, runtime_threads) are read again by every restarted worker. The watcher ignores .venv but not a virtual environment with another name, such as venv/: keep it outside the project directory or it triggers reloads. The first import of the target happens in the command itself, so an error there stops it with a message instead of waiting. dev needs watchfiles (the hinoki[server] extra) and uses one worker.
Behind a reverse proxy¶
server.trusted_proxies works as described in Hosts, CORS and HTTPS: the native layer matches the TCP peer against it for the request ID and the HSTS and scheme policy of its own answers, and the request pipeline resolves the forwarded scheme and client address of the requests it handles from the same peer. The rate limit of a natively served route resolves its client the same way, so RateLimitContext.client is the same on both lanes. Points to know:
- Entries are IP addresses or CIDRs. An IPv6 address with a zone id such as
fe80::1%lo0is refused when settings load, for either server (earlier versions loaded it and the ASGI path matched it; Kiso cannot take it). - On a listener bound to
::that also accepts IPv4 (the default on many systems, for example Linux withbindv6only=0and macOS), an IPv4 peer appears as::ffff:a.b.c.d. A plain IPv4 entry such as10.0.0.5or10.0.0.0/8then never matches, and the proxy is treated as untrusted. That fails closed, and the ASGI path behaves the same. Either bind an IPv4 host (--host 0.0.0.0) or list the mapped form, for example::ffff:10.0.0.0/104. - A request ID that a trusted proxy sent and Hinoki refused is logged as
ignored invalid request ID from trusted proxy; for requests answered natively the warning is aggregated once a second with a count.
Errors and edge cases¶
- A build whose native extension has no Kiso fails
--server kisowith an error that says so. - A socket that cannot be opened is reported as
error: cannot listen on HOST:PORT: ...with exit status2. - A worker that exits on its own is not restarted: restart policy belongs to your process manager.