Table of Contents
Network metrics of a host are captured by the SystemEdge extension of the environment’s
HpaInfrastructure agent.
They are found under SystemEdge|<host>|Network|....
Example: SystemEdge|srv-calypso01|Network|Interfaces|InterfaceDetails|enp1s0f1:In Octets 64-bit
Warning: metrics are collected every 15 s. A sample collected late covers a little more than 15 s, which can rarely produce a value slightly above the link capacity.
Warning: the first sample after a restart of the agent can be abnormally high and must be ignored.
Interfaces
Each network interface known to Linux gets its own branch. Names follow the Linux predictable naming:
| Interface | Meaning |
|---|---|
enp1s0f0 |
Physical Ethernet port: en Ethernet, p1 PCI bus 1, s0 slot 0, f0 first port of the card |
enp1s0f1 |
Second port (f1) of the same card |
enp1s0f1.2008 |
VLAN 2008 (802.1Q) sub-interface built on enp1s0f1. Not a card: its traffic is also counted on enp1s0f1 |
lo |
Loopback (127.0.0.1): traffic of the host with itself. Never leaves the machine |
docker0, br-*, tun0, wlp* |
Docker bridge, VPN tunnel, Wi-Fi… |
- The load on the wire is measured on the physical interfaces. A VLAN is never added to its parent interface.
-
lois not a network load. A high value only means many local exchanges between processes.
Metrics
Interface details
Path: SystemEdge|<host>|Network|Interfaces|InterfaceDetails|<interface>:<metric>
| Metric | Meaning | Unit | Semantics |
|---|---|---|---|
In Octets 64-bit |
Bytes received during the interval, framing included | bytes / 15 s | DELTA |
Out Octets 64-bit |
Bytes sent during the interval, framing included | bytes / 15 s | DELTA |
In Ucast Packets 64-bit, Out Ucast Packets 64-bit |
Unicast packets received / sent | packets / 15 s | DELTA |
In/Out Multicast Packets, In/Out Broadcast Packets |
Multicast / broadcast packets. Useful only to detect a broadcast storm | packets / 15 s | DELTA |
High Speed (Bits/Second) |
Negotiated link speed. Despite its name, in Mb/s (1000 = 1 Gb/s). Reference value for the utilization |
Mb/s | STATE_COUNTER |
Speed (Bits/Second) |
Same in bit/s, but 32-bit: capped at 4.29 Gb/s. High Speed is preferred |
bit/s | STATE_COUNTER |
Operation Status |
Actual link state: 1 up, 2 down, 5 dormant, 6 not present |
enum | STATE_COUNTER |
Admin Status |
Configured state (1 up, 2 down). Up while Operation Status is down = cable, switch or driver issue |
enum | STATE_COUNTER |
MTU (Octets) |
Largest packet size. Must be the same at both ends of a link | bytes | GAUGE |
The 64-bit counters are preferred. The 32-bit ones (InOctets, OutOctets) wrap every 4 GB.
Interface traffic and health
Path: SystemEdge|<host>|Network|Interfaces|<IP address>:<metric>
These metrics are stored per IP address, not per interface name. The mapping between addresses
and interfaces is not exposed by the agent. It is given on the host by the Linux command
ip -br addr, which lists each interface with its IP addresses.
| Metric | Meaning | Unit | Semantics |
|---|---|---|---|
InDiscards |
Packets received without error but dropped by the host, typically because its buffers are full. Main sign of reception saturation | packets / 15 s | DELTA |
OutDiscards |
Packets dropped before sending, typically because the output queue is full | packets / 15 s | DELTA |
InErrors, OutErrors |
Packets lost because of transmission errors. Points at hardware: cable, port, card, switch | packets / 15 s | DELTA |
OutQueueLength (Packets) |
Length of the output queue. Growing = the interface cannot keep up | packets | STATE_COUNTER |
Traffic Incoming (Packets/Sec), Traffic Outgoing (Packets/Sec) |
Packet rate. A host can saturate in packets/s (CPU, card) before saturating in bytes | packets / s | GAUGE |
TCP
Path: SystemEdge|<host>|Network|TCP:<metric>. Whole host, all interfaces together.
| Metric | Meaning | Unit | Semantics |
|---|---|---|---|
Segments Sent |
TCP segments sent, retransmissions excluded | segments / 15 s | DELTA |
Segments Retransmitted |
TCP segments sent again because they were not acknowledged in time. Packets are lost somewhere on the path, possibly outside the host | segments / 15 s | DELTA |
Segments Received, Segments Receive Error |
Segments received, and those rejected (bad checksum…) | segments / 15 s | DELTA |
EstablishedConnections |
Open TCP connections (ESTABLISHED + CLOSE-WAIT) |
connections | STATE_COUNTER |
ResetConnections |
Established connections closed abruptly | per 15 s | GAUGE |
AttemptFailures |
Failed connection attempts: refused, unreachable or timed out | per 15 s | GAUGE |
Active Opens, Passive Opens |
Connections opened by the host / accepted by the host | per 15 s | DELTA |
Resets Sent |
Segments with the RST flag sent: the host refuses or kills connections | per 15 s | DELTA |
UDP, IP routing and IP mapping metrics are also collected (Network|UDP, Network|IP|...). They
are rarely useful for Calypso and are not described here.
Use cases
Is the link saturated?
Utilization of one direction of a physical interface, from In Octets 64-bit (or Out) and
High Speed:
Utilization (%) = Octets x 8 / (15 x High Speed x 1 000 000) x 100 = Octets / (18 750 x High Speed)
Each direction is computed separately: a link is full-duplex.
Link (High Speed) |
50 % | 75 % | 100 % |
|---|---|---|---|
100 Mb/s (100) |
93.75 MB / 15 s | 140.6 MB / 15 s | 187.5 MB / 15 s |
1 Gb/s (1000) |
937.5 MB / 15 s | 1.41 GB / 15 s | 1.875 GB / 15 s |
Sustained utilization above 75 % is a saturation risk. 15 s averages hide shorter bursts: an interval at 60 % very likely contains seconds at 100 %.
Example: a host with a public interface at 1 Gb/s and a private interface at 100 Mb/s carrying the database traffic. The public interface stays below 7 %, while the private one averages 20 % with daily peaks between 53 % and 100 %. The private link is the bottleneck, 10 times slower than the public one while it carries almost all the useful traffic.
Does the host suffer?
Utilization alone does not prove a slowdown. Drops and retransmissions are checked over the same period:
-
InDiscards/OutDiscardson the IP addresses of the physical interfaces: any increase during a traffic peak means saturation. -
InErrors/OutErrors: not a load issue. An increase calls for a check of the cable, the switch port and the link speed. -
Retransmission rate = sum of
Segments Retransmitted/ sum ofSegments Sent. Below 0.5 % is normal. Above 1 %, packets are lost on the path.
- A saturated link does not always produce drops on the host: TCP slows the senders down instead. Discards can stay at 0 for weeks on a link regularly full. Utilization remains the main indicator, and the absence of drops is not proof of good health.
- Correlating a retransmission peak with a traffic peak points at the interface involved.
Is the link in the expected state?
-
Operation Statusmust stay at1. Any other value is an outage of the interface. -
High Speedmust match the expected speed. A 1 Gb/s card shown at100usually means a failed auto-negotiation (cable, switch port) or an offer limited by the hosting provider. It is checked on the host with the Linux commandethtool <interface>:Speedgives the negotiated speed,Supported link modesthe speeds the card can reach.
TCP activity
-
EstablishedConnections: baseline of the activity. A sudden change reveals a new client, a connection leak or a restart. -
ResetConnectionsandAttemptFailures: a sudden increase reveals a server that refuses or drops connections (restart, exhausted pool, firewall). -
Traffic Incoming/Outgoing (Packets/Sec): a high packet rate with a low byte rate means many small packets, which load the CPU more than the link.
Reading the data correctly
| Situation | Symptom | Action |
|---|---|---|
| Agent restart | One isolated, huge point (hundreds of GB) | Filtered out with a value selector, e.g. LT the 100 % value of the link |
| Late collection | A 15 s point slightly above 100 % of the link | Capped at 100 %, or reasoning on 1 min to 1 h averages |
| Collection gap | No points over a period | The agent status is checked before concluding there was no traffic |
| VLAN and parent | VLAN traffic close to its parent interface | The two are never added together |