Onload-9.1.0
============

 This is a feature release of Onload adding support for the
 AMD Solarflare(TM) X4 Series Ethernet Adapter. The release
 includes a new version of the sfc net driver and various
 features and fixes.

 Please see the ChangeLog for a full list of changes.


Linux distribution support
--------------------------

 This package can be installed on:
 - Red Hat Enterprise Linux 8.10
 - Red Hat Enterprise Linux 9.4 - 9.7(*)
 - Red Hat Enterprise Linux 10.0 - 10.1(*)
 - Canonical Ubuntu Server LTS 22.04 and 24.04
 - Debian 12 "Bookworm"
 - Debian 13 "Trixie"
 - Linux kernels 5.11 - 6.14

(*) At the time of writing, RHEL 9.7 and RHEL 10.1 were in a pre-release state
    but expected to be compatible with this Onload version.


AMD Solarflare(TM) X4 Series Ethernet Adapter
---------------------------------------------

This release adds support for the X4 adapter.

- NIC reset: A driver reload is required following an adapter reset.
- Filter re-insertion when interfaces change state is not supported.
- Shared queues (shrub) have associated recommendations for effective use.

X4 Datapath Restrictions
------------------------

- The Wire Order API (WODA) is only guaranteed when only the Enterprise RX path
  is selected (EF_RX_DATAPATH=enterprise)
- Application clustering (SO_REUSEPORT/EF_CLUSTER_SIZE) is only supported with
  EF_RX_DATAPATH=enterprise
- Sharing an event queue between multiple VIs is not supported when using the
  express datapath. This applies to X3 and X4 network cards.

X4 Diagnostics
--------------

The onload_shrubdump tool provides diagnostic information from shared
reusable buffer controllers. Its output format and debug logging may change
in future releases.

X4 Known Issues
---------------

- The X4 enterprise datapath does not differentiate between CRC errors and
  truncation errors in RX discard events. As a result, these will be reported
  as EF_EVENT_RX_DISCARD_CRC_BAD in EF_EVENT_TYPE_RX_DISCARD for ef_vi users.
  This will be visible to onload users via the rx_discard_crc_bad statistic in
  stackdump. A side effect of this change is that truncation errors may be
  mischaracterised as CRC errors, including on older network cards.


AMD Alveo(TM) X3 Series Low Latency Ethernet Adapter
----------------------------------------------------

The minimum compatible X3522 driver version to match the Onload supported OS
list is v1.6.8.0.


New ef_receive_poll() API call
------------------------------

In recent releases, the ef_vi API gained a facility to poll for receives
separately from transmissions, for potential latency improvements in some
situations. API usage of ef_receive_poll() is identical to that of
ef_eventq_poll() although usage should be gated by a check for support by
the capabilities API (EF_VI_CAP_RX_POLL), for example:

    rc = ef_vi_capabilities_get(driver_handle, ifindex, EF_VI_CAP_RX_POLL,
                                &can_rx_poll);

    if( rc == 0 && can_rx_poll )
      num_events = ef_receive_poll(vi, events, max_events);
    else
      num_events = ef_eventq_poll(vi, events, max_events);

    for( i = 0; i < num_events; i++ ) {
      switch( EF_EVENT_TYPE(events[i]) ) {
        /* handle rx events */
      }
    }

See also the source code for eflatency which uses this feature.


Smart Data Cache Injection
--------------------------

Smart Data Cache Injection (SDCI) support has been added to allow steering of
network packets directly to a core's L2 cache. Currently, SDCI is limited to
X2/X4 NICs and must be configured as documented below. The steering tag used is
such that the data is steered to the core complex that the VI is created on.

SDCI support in the sfc net driver requires a kernel with TPH support
(i.e., CONFIG_PCIE_TPH=y), otherwise the request for SDCI support will be
overriden. To verify whether SDCI support is enabled after compilation, check
the compilation log for lines of the format:

    Kernel does not support PCIE_TPH
    TPH functionality will be disabled

Also be aware that use of an onload with SDCI support running on a machine that
lacks CPU support will result in verbose logging from the sfc net driver when
it fails to get the steering tags for each of its channels. This will result
in messages of the following format being added to the kernel log (dmesg):

    pcie_tph_get_cpu_st error(-22) channel <intf-name>-<channel>, cpu <cpu>

On systems where SDCI is not supported by the PCIe root port the error logged
is:

    failed enabling TPH support

To use this feature with onload, the EF_TPH_MODE environment variable should be
used, which has the following valid values:

 - Disable SDCI (default)           : EF_TPH_MODE=0
 - Enable SDCI without steering tags: EF_TPH_MODE=1
 - Enable SDCI with steering tags   : EF_TPH_MODE=2

To use this feature with ef_vi, one can provide the following VI flags at VI
allocation time:

 - Enable SDCI without steering tags: EF_VI_ENABLE_TPH
 - Enable SDCI with steering tags   : (EF_VI_ENABLE_TPH | EF_VI_TPH_TAG_MODE)

A more coarse-grained solution also exists at the application level, which will
apply to all VIs, using the EF_VI_EXTRA_VI_FLAGS environment variable:

 - Enable SDCI without steering tags: EF_VI_EXTRA_VI_FLAGS=tph_enable
 - Enable SDCI with steering tags   : EF_VI_EXTRA_VI_FLAGS=tph_enable,tph_mode

Please note that when using SDCI with steering tags, a NIC reset may result in
incorrect steering tags being used, which may degrade performance.


X2 Rx Timestamping with v7.1 firmware or older
----------------------------------------------

Using firmware version v7.1 or older with this release will lead to
inaccurate timestamps. Updating to a recent firmware version addresses this.


Indirect Branch Tracking (IBT) Incompatibility
----------------------------------------------

Onload is not compatible with IBT. To use Onload with kernels which include IBT
support (e.g. RHEL10), ibt=off should be appended to the kernel commandline.
Attempting to load the onload drivers when IBT is still enabled will result in
failure to insert the sfc_resource kernel module. Onload will also output a
message to the kernel log suggesting ibt=off is appended to the kernel
commandline.


New package for example applications
------------------------------------

Source code examples showcasing how to build applications to use certain
features of Onload and ef_vi are now included in a new optional subpackage
called 'onload-examples'. The examples were previously available in Onload
tarball source release packages and could be extracted from RPM and Debian
source packages.


DKMS RPM no longer includes userspace and is built from SRPM bundle
-------------------------------------------------------------------

To use DKMS, build and install the DKMS RPM and userspace RPM sub-packages
together from the Source RPM.


Deprecation of tarball installation method (removal under consideration)
------------------------------------------------------------------------

A future feature release MAY drop the provision of an OS-independent
'tarball' release package. In the meantime, the OS-specific packages (source
RPM and source DEB) are recommended for the best installation experience.
Docker images are also available via the Onload Operator v3.0 for Kubernetes.

Please contact your sales or support representative if you have any feedback
on this proposed change in direction for packaging support.


Transmit error recovery improvements
------------------------------------

Transmit error events, reported at the ef_vi level via EF_EVENT_TYPE_TX_ERROR,
benefit from improved support. Onload is now able to recover from such events
and ef_vi is able to handle these events without needing to entirely recreate
the VI that experienced the TX error event. This is only supported on the X4
express datapath.

Recovery from such events in onload is enabled by default, and can be disabled
by setting EF_ENABLE_TX_ERROR_RECOVERY=0. Handling of TX error events in ef_vi
is described and documented in the API functions:
- ef_vi_reinit_txq_post_error
- ef_vi_for_each_tx_error_failed_transmit


Change in behaviour for CRC and truncation RX discards
------------------------------------------------------

To support X4, which is unable to differentiate between these error types in RX
discard events, any NIC using the ef10 architecture, X2 or SFN8k, will also not
differentiate between these error types. This only affects users with an MTU of
larger than 1500.


(c) Copyright 2025 Advanced Micro Devices, Inc.
