Merged
Conversation
The TaskEvent API was removed in favor of a future OpenTelemetry integration. The README should not document APIs that no longer exist in the codebase.
Add new top-level sections for Tasks (dataclass, nested tracking, proxy serialization, two-layer serialization) and Workers (gRPC server, dedicated execution thread, bidirectional streaming). Expand Routines with dispatch gate and coroutine vs async generator semantics, Discovery with events and protocol internals, Load balancing with error classification and worker connections, Security with TLS modes and discovery filtering, and Error handling with exception transmission and version compatibility. Add stable pip install command. Remove task lifecycle events section.
Covers channel pooling lifecycle, reactive error detection via gRPC/HTTP2, and the absence of keepalive ping configuration.
LocalWorker.__init__ was eagerly resolving WorkerCredentials properties into grpc.ServerCredentials and grpc.ChannelCredentials — C extension types that cannot be pickled. On spawn-based platforms (macOS), multiprocessing.Process.start() pickles the WorkerProcess, causing a TypeError. Drop @Property from WorkerCredentials credential methods, store the picklable dataclass through the subprocess boundary, and resolve gRPC credential objects only at the point of use in _serve() and _stop(). BREAKING CHANGE: WorkerCredentials.server_credentials and client_credentials are now methods (call with parentheses) instead of properties. WorkerProcess accepts credentials (WorkerCredentials | None) instead of server_credentials (ServerCredentialsType).
…tate The _index dict accumulates LoadBalancerContext → WorkerConnection references containing unpicklable grpc.ChannelCredentials objects. When WorkerProxy.__reduce__ pickles a live lb instance for nested dispatch, this causes a TypeError. The connection state is dead weight — the restored proxy creates a fresh context and re-discovers workers from scratch.
WorkerCredentials gains a module-level ContextVar, __enter__/__exit__ methods, and a current() classmethod. Workers set their credentials into this ContextVar on startup so that unpickled proxies can resolve peer credentials without serializing private key material. BREAKING CHANGE: _token field added to frozen dataclass (excluded from init, repr, compare, and equality — no behavioral break for existing callers).
Credentials now live in WorkerCredentials.current() via ContextVar, so RuntimeContext no longer needs to carry them. This separates concerns ahead of RuntimeContext becoming a task-level dispatch settings container. BREAKING CHANGE: RuntimeContext no longer accepts a credentials parameter or exposes a credentials property.
Wraps the entire server lifecycle in the WorkerCredentials context manager so that nested proxies created during task execution can resolve peer credentials from WorkerCredentials.current() instead of requiring serialized credentials.
WorkerProxy now defaults to Undefined credentials and resolves them from WorkerCredentials.current() at construction time. __reduce__ serializes options but omits credentials entirely — the restored proxy re-resolves from the worker's ContextVar, implementing peer authentication without credential delegation. WorkerPool passes WorkerCredentials directly to WorkerProxy instead of pre-resolving client credentials. WorkerConnection now accepts grpc.ChannelCredentials directly. Removes ChannelCredentialsType and resolve_channel_credentials from base.py — the callable-credential indirection layer is no longer needed now that WorkerCredentials.client_credentials() is the sole resolution path. BREAKING CHANGE: WorkerProxy credentials parameter type changed from ChannelCredentialsType to WorkerCredentials | None | UndefinedType. WorkerPool credentials parameter type changed from WorkerCredentials | None | UndefinedType to WorkerCredentials | None. ChannelCredentialsType and resolve_channel_credentials removed.
Define a RuntimeContext message with a dispatch_timeout field and add it as an optional submessage on Task (field 11). The optional qualifier enables HasField checks so workers can distinguish absent context (older clients) from context with default values. Re-export the new RuntimeContext type from the protocol package.
RuntimeContext gains to_protobuf/from_protobuf methods that serialize the wire-safe subset of the context (dispatch_timeout only; credentials are TLS material and intentionally excluded). The proto default 0.0 maps to None in Python. Task gains a context field captured automatically in __post_init__ via RuntimeContext.get_current(). The field is serialized as an optional submessage on the wire Task and deserialized with a HasField check for backward compatibility with older clients. _run() and _stream() now enter self.context as a context manager before calling the user's function, restoring dispatch_timeout into the ContextVar so nested dispatch respects the original timeout. The existing __enter__/__exit__ token machinery on RuntimeContext ensures sequential tasks on the same worker loop do not leak state.
…pagation Cover RuntimeContext.to_protobuf/from_protobuf with example-based and property-based (Hypothesis) roundtrip tests. Cover Task context capture in __post_init__, protobuf roundtrip with context submessage, backward compatibility for messages without context, and dispatch_timeout restoration in both _run() and _stream() including sequential task isolation.
The context manager protocol (__enter__/__exit__) and the current() classmethod on WorkerCredentials existed solely for internal ContextVar propagation in worker subprocesses. Exposing them publicly invited misuse where users could override credentials outside of WorkerPool, causing silent divergence between pool-configured and ContextVar credentials. Introduce _CredentialContext — an internal helper that owns the ContextVar lifecycle — and strip the CM protocol, current(), and the _token field from WorkerCredentials. Update WorkerProcess._serve() and WorkerProxy.__init__() to use the new helper.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Auto-generated by the cut release workflow.