Espex.SerialProxy behaviour (espex v0.12.0)

Copy Markdown View Source

Behaviour for serial proxy adapters.

Implement this module to expose one or more serial ports to ESPHome clients through the Native API's serial proxy feature. Home Assistant can then talk to the port over the network as though it were local — useful for Zigbee coordinators, CLI debug ports, and similar "tunnel a UART through the device" use cases.

Espex itself owns no port state. It calls your callbacks from the per-connection handler process when a client configures, writes to, or closes an instance.

Callbacks

CallbackRequiredPurpose
list_instances/0yesAdvertise the available ports
open/3yesOpen an instance with the client's requested UART params
write/2yesWrite bytes to an opened instance
close/1yesClose and release an instance
set_modem_pins/3noToggle RTS/DTR
get_modem_pins/1noRead RTS/DTR
request/2noHandle subscribe / unsubscribe / flush
default_open_opts/1noOptions for an espex-initiated lazy open
set_mode/2noSwitch the port between RAW and PROTOCOL mode
identity/1noIdentify the device behind a port (API 1.18)

Omitted optional callbacks degrade gracefully: request/2 operations are answered with a NOT_SUPPORTED status, get_modem_pins/1 reports all lines low, set_modem_pins/3 becomes a no-op, default_open_opts/1 falls back to default_open_opts/0 (9600-8-N-1), and set_mode/2 answers NOT_SUPPORTED for PROTOCOL (RAW is always OK), identity/1 reports every port with source NONE — your adapter can safely omit whatever the hardware can't do.

Ownership

Since 0.11.0 (API 1.17) each instance has at most one owner: the connection that sent SUBSCRIBE for it. This is ESPHome's single-owner rule, and it applies even when no one is subscribed yet — a client must SUBSCRIBE before anything else:

Request from a non-ownerResult
CONFIGURE, SET_MODEM_PINS, FLUSH, SET_MODEacknowledged PORT_IN_USE, adapter not called
WRITEdropped silently (WRITE has no acknowledgement)
GET_MODEM_PINSanswered — it is read-only and ungated upstream
SUBSCRIBE while another live connection owns itacknowledged PORT_IN_USE
UNSUBSCRIBEacknowledged OK (idempotent)

Ownership is released by UNSUBSCRIBE and when the owning connection closes for any reason. Both close the owner's handle (close/1) before the release, so once an instance shows as free its port really is: an adapter that allows one open per device works with the handoff. The one exception is a connection killed outright, which espex cannot tear down; its ownership is still swept, so a handle must not outlive the subscriber pid it was opened for (see open/3). A recorded owner whose connection process has already exited is taken over by the next SUBSCRIBE. A peer that vanished without closing its socket keeps its process, and so its ownership, until the keepalive reaps it (about two minutes with the defaults), and a reconnecting client is PORT_IN_USE for that window — the same as on firmware. Ownership is tracked across connections by Espex.Server, one record per instance.

The rule gates requests and data, not handles: espex still opens a handle per connection via open/3, so a non-owner's GET_MODEM_PINS may lazily open a second handle on the same instance. Whether that second open is allowed is the adapter's decision, as before; bytes the adapter delivers on such a handle are dropped, only the owner receives SerialProxyDataReceived.

Port mode

API 1.17 adds SerialProxySetModeRequest with two modes. RAW (the default) forwards bytes untouched. PROTOCOL asks the device to run a protocol handler — a tap in ESPHome terms — on the port instead of, or alongside, the raw stream. Espex has no tap object of its own: the adapter's optional set_mode/2 is the whole surface, and an adapter that does not export it answers NOT_SUPPORTED to PROTOCOL.

The mode belongs to the owner's session, per instance. Espex reapplies a PROTOCOL mode after a CONFIGURE reopens the port (the fresh handle starts raw). The mode dies with the handle: both UNSUBSCRIBE and disconnect close it without a set_mode/2 call, so close/1 is where an adapter tears down its protocol handler. set_mode/2 with :raw only arrives when a client asks for it explicitly.

Data flow

After a successful open/3, data arriving on the port must be sent to the subscriber pid (the per-connection handler) as:

{:espex_serial_data, handle, binary}

handle is the opaque term you returned from open/3 — typically a pid, reference, or small tuple. The handler uses it to correlate the data back to an instance id.

The client can later change port parameters by issuing another configure request; the connection handler will close/1 the existing handle and call open/3 again with the new options. Your adapter doesn't need to handle reconfigure-in-place.

open_opts reference

open/3 receives a keyword list shaped like open_opts/0:

KeyValuesDefault when the client sends 0
:speedbaud rate9600
:data_bits5, 6, 7, 88
:stop_bits1, 21
:parity:none, :even, :odd:none
:flow_control:none, :hardware:none

The defaults follow common "9600-8-N-1" convention and match what ESPHome uses when fields are omitted from a SerialProxyConfigureRequest. Use configure_request_to_open_opts/1 if you're building your own message-routing test harness; the connection handler calls it for you in normal operation.

SerialProxy.Info for advertisements

list_instances/0 returns a list of Espex.SerialProxy.Info structs, one per port. Each needs:

  • instance — stable non_neg_integer() id; the client refers to ports by this id in all subsequent requests
  • name — display name shown in the Home Assistant UI
  • port_type — :ttl, :rs232, :rs485, or :usb_serial (a USB serial adapter, since 0.12.0)
  • configured_line_states — the modem control lines this port can drive, a list of :rts / :dtr (default []). Advertised to the client as a bitmask so it knows which pins set_modem_pins/3 will honour. Set it when the adapter implements set_modem_pins/3. Since 0.10.0.

The list is snapshotted at connection-accept time and cached by the client; see the "Architecture" guide for why changes require a reconnect.

Identity

Since 0.12.0 (API 1.18) a client can subscribe to the identity of the device behind each port. espex answers with one Espex.SerialProxy.Identity per advertised port, from identity/1, then forwards every change you push:

Espex.push_serial_proxy_identity(server_name, %Espex.SerialProxy.Identity{
  instance: 0,
  source: :usb,
  connected?: true,
  manufacturer: "Nabu Casa",
  product: "ZBT-2",
  serial_number: "3c2f19a0",
  usb: %{vendor_id: 0x303A, product_id: 0x4001}
})

Push a change when a USB device is attached to or removed from a port. Each connection forwards it only if it subscribed and advertises that instance. Without identity/1, every port reports source :none.

Acknowledgements

Since 0.10.0 (API 1.16), every request except WRITE is answered with a SerialProxyRequestResponse (or, for GET_MODEM_PINS, a SerialProxyGetModemPinsResponse carrying status). An unknown instance is always INVALID_ARGUMENT, checked before ownership; a non-owner is PORT_IN_USE (see "Ownership"):

RequestStatus
CONFIGUREOK when open/3 succeeds, ERROR with the reason when it fails
SET_MODEM_PINSOK / ERROR from set_modem_pins/3, NOT_SUPPORTED when the adapter does not implement it or returns {:error, :not_supported}
GET_MODEM_PINSOK with the line states, NOT_SUPPORTED, or ERROR
SUBSCRIBEOK once the instance is claimed — the adapter's request/2 answer is logged, not echoed, since ownership is taken regardless; PORT_IN_USE when another live connection owns it
UNSUBSCRIBEOK once the session is torn down (the adapter hears :unsubscribe, the handle is closed, ownership is released) — like SUBSCRIBE, the request/2 answer is logged, not echoed; always OK for a non-owner
FLUSHthe status returned by request/2
SET_MODEOK / ERROR from set_mode/2; RAW is OK even while the port is not open; NOT_SUPPORTED for PROTOCOL when the adapter does not implement it or returns {:error, :not_supported}; INVALID_ARGUMENT for a mode outside the enum (checked after ownership, as upstream: a non-owner is PORT_IN_USE whatever it sent)

A SUBSCRIBE that arrives before the port is open is acknowledged OK as soon as the claim succeeds and the intent is recorded — not once the port is open. The lazy open it triggers may fail; the intent survives and is reattached on the next successful open (CONFIGURE), which is how ESPHome firmware behaves too, where subscribe always succeeds because the UART is always live.

A lazy open (the owner's WRITE, FLUSH, SET_MODEM_PINS or SET_MODE, or any connection's GET_MODEM_PINS, arriving before any CONFIGURE) is not itself acknowledged; only the request that triggered it is.

Example: a port wrapping Circuits.UART

The following sketch wires a single port (/dev/ttyUSB0) to a Zigbee-style advertisement. It assumes a real Circuits.UART-like library is available; adapt to whichever serial library you use.

defmodule MyApp.SerialAdapter do
  @behaviour Espex.SerialProxy

  @impl true
  def list_instances do
    [Espex.SerialProxy.Info.new(instance: 0, name: "zigbee", port_type: :ttl)]
  end

  @impl true
  def open(0, opts, subscriber) do
    {:ok, pid} = MyApp.SerialPort.start_link(
      device: "/dev/ttyUSB0",
      subscriber: subscriber,
      speed: opts[:speed],
      data_bits: opts[:data_bits],
      stop_bits: opts[:stop_bits],
      parity: opts[:parity],
      flow_control: opts[:flow_control]
    )
    {:ok, pid}
  end

  def open(_unknown, _opts, _subscriber), do: {:error, :no_such_instance}

  @impl true
  def write(pid, data), do: MyApp.SerialPort.write(pid, data)

  @impl true
  def close(pid) do
    _ = MyApp.SerialPort.stop(pid)
    :ok
  end

  @impl true
  def set_modem_pins(pid, rts, dtr), do: MyApp.SerialPort.set_pins(pid, rts: rts, dtr: dtr)

  @impl true
  def get_modem_pins(pid), do: MyApp.SerialPort.get_pins(pid)

  @impl true
  def request(pid, :flush), do: MyApp.SerialPort.flush(pid)
  def request(_pid, _), do: {:ok, :not_supported}
end

And inside the wrapper GenServer (MyApp.SerialPort), any time your read loop gets a chunk from the OS, forward it to the subscriber:

send(state.subscriber, {:espex_serial_data, self(), chunk})

request/2 semantics

The optional request/2 callback handles the three operations the ESPHome wire protocol exposes:

  • :subscribe / :unsubscribe — espex already wires data delivery at open/3 time via the subscriber pid. Most adapters can return {:ok, :not_supported} or {:ok, :ok} here. Implement them only if your adapter keeps a separate stream-enabled flag you want the client to toggle explicitly.
  • :flush — block until all queued TX data has been drained. Return {:ok, :ok} if you confirmed the drain, {:ok, :assumed_success} if the platform can't report completion, {:ok, :timeout} if you gave up waiting, or {:error, reason} on failure.

When you don't define request/2 at all, espex responds with :not_supported to every request type automatically.

Upstream ESPHome has no open/close lifecycle for the serial proxy — the UART is always live at its YAML-configured settings, and SerialProxyConfigureRequest is optional re-tuning, not a prerequisite. Espex mirrors that: the first operation that needs a port on an advertised instance lazily opens it via open/3, using default_open_opts/1 when you export it (or the 9600-8-N-1 fallback otherwise). SUBSCRIBE acquires ownership and is that first operation in practice; WRITE, FLUSH, SET_MODEM_PINS and SET_MODE require the ownership it acquired (see "Ownership"); GET_MODEM_PINS opens without it; UNSUBSCRIBE never opens. This lets a client resume traffic after a reconnect — subscribe, then write to a Zigbee coordinator without re-sending CONFIGURE — instead of getting every write silently dropped.

SUBSCRIBE/UNSUBSCRIBE track a per-connection subscribe intent rather than a one-shot stash: after every successful open/3 (whether triggered by a lazy open or by CONFIGURE), espex re-issues request/2 with :subscribe if the intent is set, so a subscription survives reconfiguration instead of being consumed by the first open. Your adapter still never receives request/2 before open/3 — the handle argument is always one you just returned. A :subscribe or :unsubscribe that can't reach the adapter yet (the lazy/configure open failed) is still acknowledged OK; the intent is remembered and reattached on the next successful open. :flush keeps its original semantics; if the lazy open itself fails the client receives an ERROR response (error_message: ":not_open").

A failed lazy open is retried at most once per second per instance; an explicit CONFIGURE always attempts an open, and a successful open resets the backoff.

Wiring

Pass your adapter module to Espex.start_link/1:

Espex.start_link(
  device_config: [name: "serial-gateway"],
  serial_proxy: MyApp.SerialAdapter
)

Summary

Types

Which request a SerialProxyRequestResponse acknowledges: any request_type/0, or CONFIGURE / SET_MODEM_PINS / SET_MODE, which are separate messages and never reach request/2.

Opaque handle returned by the adapter from open/3.

Port mode selected by a SerialProxySetModeRequest (API 1.17): :raw forwards bytes untouched; :protocol asks the adapter to run its own protocol handling on the port. See set_mode/2.

Options for opening a serial port. Keys follow the fields on SerialProxyConfigureRequest but normalised to atoms.

Internal atom form of the SerialProxyStatus enum.

Internal atom form of the SerialProxyRequestType values a client can send in a SerialProxyRequest; the operations request/2 receives.

Callbacks

Close an opened instance and release any associated resources.

Return the options espex should use when it opens instance lazily — i.e. when a client sends a write/subscribe/modem-pins/flush request without a prior SerialProxyConfigureRequest on this connection.

Read the current state of the RTS and DTR modem control lines. Optional — return {:error, :not_supported} if the adapter doesn't support modem pin control.

The current identity of the device behind instance, or nil when the port carries none (reported with source :none). Optional. espex calls it when a client subscribes to identities (API 1.18); push later changes with Espex.push_serial_proxy_identity/2. See "Identity" above.

Return the list of available serial proxy instances.

Open the given instance with the supplied options. Data received on the port must be forwarded to subscriber as {:espex_serial_data, handle, binary}.

Handle one of the SerialProxyRequest operations (subscribe, unsubscribe, flush) and return a status for the client. Optional — when undefined, espex responds with :not_supported.

Switch an opened instance between :raw and :protocol mode (SerialProxySetModeRequest, API 1.17). Optional.

Set the RTS and DTR modem control lines. Optional — return {:error, :not_supported} if the adapter doesn't support modem pin control.

Write bytes to an opened instance.

Functions

Translate a SerialProxyConfigureRequest protobuf into the keyword list passed to open/3. Zero-valued protobuf fields fall back to sensible defaults (9600-8-N-1, no flow control).

The fallback options used when espex lazily opens an instance and the adapter does not export default_open_opts/1: 9600-8-N-1, no flow control (the same defaults used for zero-valued fields in a SerialProxyConfigureRequest).

Types

ack_type()

@type ack_type() :: request_type() | :configure | :set_modem_pins | :set_mode

Which request a SerialProxyRequestResponse acknowledges: any request_type/0, or CONFIGURE / SET_MODEM_PINS / SET_MODE, which are separate messages and never reach request/2.

handle()

@type handle() :: term()

Opaque handle returned by the adapter from open/3.

mode()

@type mode() :: :raw | :protocol

Port mode selected by a SerialProxySetModeRequest (API 1.17): :raw forwards bytes untouched; :protocol asks the adapter to run its own protocol handling on the port. See set_mode/2.

open_opts()

@type open_opts() :: [
  speed: non_neg_integer(),
  data_bits: 5..8,
  stop_bits: 1..2,
  parity: :none | :even | :odd,
  flow_control: :none | :hardware
]

Options for opening a serial port. Keys follow the fields on SerialProxyConfigureRequest but normalised to atoms.

request_status()

@type request_status() ::
  :ok
  | :assumed_success
  | :error
  | :timeout
  | :not_supported
  | :port_in_use
  | :invalid_argument

Internal atom form of the SerialProxyStatus enum.

request_type()

@type request_type() :: :subscribe | :unsubscribe | :flush

Internal atom form of the SerialProxyRequestType values a client can send in a SerialProxyRequest; the operations request/2 receives.

Callbacks

close(handle)

@callback close(handle()) :: :ok

Close an opened instance and release any associated resources.

default_open_opts(instance)

(since 0.8.0) (optional)
@callback default_open_opts(instance :: non_neg_integer()) :: open_opts()

Return the options espex should use when it opens instance lazily — i.e. when a client sends a write/subscribe/modem-pins/flush request without a prior SerialProxyConfigureRequest on this connection.

On real ESPHome hardware the UART is always live with its YAML-configured settings, so clients legitimately resume traffic after a reconnect without re-configuring. Implement this to supply your port's natural settings (e.g. the baud rate the attached device actually uses). When not exported, espex falls back to default_open_opts/0 (9600-8-N-1).

get_modem_pins(handle)

(optional)
@callback get_modem_pins(handle()) ::
  {:ok, %{rts: boolean(), dtr: boolean()}} | {:error, term()}

Read the current state of the RTS and DTR modem control lines. Optional — return {:error, :not_supported} if the adapter doesn't support modem pin control.

identity(instance)

(since 0.12.0) (optional)
@callback identity(instance :: non_neg_integer()) :: Espex.SerialProxy.Identity.t() | nil

The current identity of the device behind instance, or nil when the port carries none (reported with source :none). Optional. espex calls it when a client subscribes to identities (API 1.18); push later changes with Espex.push_serial_proxy_identity/2. See "Identity" above.

list_instances()

@callback list_instances() :: [Espex.SerialProxy.Info.t()]

Return the list of available serial proxy instances.

open(instance, open_opts, subscriber)

@callback open(instance :: non_neg_integer(), open_opts(), subscriber :: pid()) ::
  {:ok, handle()} | {:error, term()}

Open the given instance with the supplied options. Data received on the port must be forwarded to subscriber as {:espex_serial_data, handle, binary}.

subscriber is the connection that owns the handle. Espex calls close/1 on every orderly teardown, but a connection killed with an untrappable exit never reaches that call, while its ownership is still swept so the next client can claim the instance. A handle that outlives its subscriber must therefore watch that pid (Process.monitor/1 or a link) and close itself when it goes down; an exclusive-open adapter would otherwise refuse the next owner's open.

request(handle, request_type)

(optional)
@callback request(handle(), request_type()) :: {:ok, request_status()} | {:error, term()}

Handle one of the SerialProxyRequest operations (subscribe, unsubscribe, flush) and return a status for the client. Optional — when undefined, espex responds with :not_supported.

Subscribe/unsubscribe are currently no-ops in the default espex flow (data delivery is wired at open/3 time); adapters that care about explicit stream control can implement the toggle here.

set_mode(handle, mode)

(since 0.11.0) (optional)
@callback set_mode(handle(), mode()) :: :ok | {:error, :not_supported} | {:error, term()}

Switch an opened instance between :raw and :protocol mode (SerialProxySetModeRequest, API 1.17). Optional.

Return :ok when the mode is in effect, {:error, :not_supported} when the port has no protocol handler to enable (acknowledged NOT_SUPPORTED), or {:error, reason} for a failure (acknowledged ERROR). Espex calls this with :raw only when the owner leaves protocol mode explicitly, and re-issues :protocol on the new handle after a CONFIGURE reopens the port. A session ending (UNSUBSCRIBE or disconnect) closes the handle instead; tear the protocol handler down in close/1. See "Port mode" above.

set_modem_pins(handle, rts, dtr)

(optional)
@callback set_modem_pins(handle(), rts :: boolean(), dtr :: boolean()) ::
  :ok | {:error, term()}

Set the RTS and DTR modem control lines. Optional — return {:error, :not_supported} if the adapter doesn't support modem pin control.

write(handle, data)

@callback write(handle(), data :: binary()) :: :ok | {:error, term()}

Write bytes to an opened instance.

Functions

configure_request_to_open_opts(req)

@spec configure_request_to_open_opts(Espex.Proto.SerialProxyConfigureRequest.t()) ::
  open_opts()

Translate a SerialProxyConfigureRequest protobuf into the keyword list passed to open/3. Zero-valued protobuf fields fall back to sensible defaults (9600-8-N-1, no flow control).

default_open_opts()

(since 0.8.0)
@spec default_open_opts() :: open_opts()

The fallback options used when espex lazily opens an instance and the adapter does not export default_open_opts/1: 9600-8-N-1, no flow control (the same defaults used for zero-valued fields in a SerialProxyConfigureRequest).