All posts
Engineering21 AUG 202611 min read

RTBH and FlowSpec: What Your Upstream Can and Cannot Do

MF

Matt F.

Network Engineering

Get an AI summary

If your uplink fills at 03:00, the first useful question is not necessarily which mitigation service you should buy.

It is what your existing network can already do.

Many transit providers expose one or both of two mechanisms through BGP: remotely triggered blackholing, or RTBH, and BGP Flow Specification, usually shortened to FlowSpec. Neither is a substitute for a mitigation platform, but both are useful, both can stop certain attacks before they reach your port, and both are frequently misunderstood.

The distinction matters because their limitations are unusually clean.

RTBH can remove a destination from the network. FlowSpec can describe and filter traffic that matches a known stateless signature. Neither can determine whether a packet belongs to a legitimate session simply because it looks plausible in isolation.

That boundary is worth understanding before adding anything more complicated.

Two tools you may already have

RTBH and FlowSpec solve related problems in very different ways.

RTBH changes what happens to traffic for an IP prefix. You advertise a route with a community your upstream recognizes as a request to discard the traffic.

FlowSpec distributes filtering policy through BGP. Instead of saying "discard everything for this destination," it can say something closer to "for traffic toward this prefix, matching this protocol, source port, packet length, or TCP flag combination, apply this action."

The first is intentionally blunt. The second is substantially more precise.

Neither is automatically available simply because you have a BGP session. The standardized BLACKHOLE community is advisory, and an operator must explicitly choose to honor it. Customer-originated FlowSpec likewise depends on the provider's routing policy and implementation.

But when your upstream supports them, these are mitigation primitives you may already have as part of the network relationship you are paying for.

RTBH, and the important part of the definition

Destination-based RTBH is simple.

Suppose 203.0.113.17 is under attack. You advertise a more-specific route covering that address, typically a /32 for IPv4 or /128 for IPv6, and attach the BLACKHOLE community. RFC 7999 standardizes that community as 65535:666.

A participating upstream recognizes the route and discards traffic destined for it, ideally at the provider's ingress edge rather than allowing the traffic to travel across its network and down your transit interface. RFC 7999 specifically recommends blackhole prefixes be made as specific as possible to minimize collateral damage.

That is useful. If one address is receiving 200 Gbps and your port is 100 Gbps, blackholing the target upstream can restore service to everything else behind that port.

It does not restore service to the target.

This is the distinction that gets blurred when RTBH is described generically as "DDoS mitigation." Destination-based blackholing completes the denial of service against the selected address. Attack traffic is dropped, but so is every legitimate packet destined for the same address.

RFC 5635 says this explicitly: destination-based RTBH makes the target unreachable while limiting collateral damage to the rest of the customer or provider network.

That makes RTBH an excellent containment mechanism.

It is not selective protection.

If sacrificing one address keeps another thousand customers reachable, that may be exactly the correct operational decision. It should simply be understood as triage rather than filtering.

Source-based RTBH is different

There is another form of RTBH that reverses the idea.

Rather than blackholing the destination being attacked, source-based RTBH installs a discard route for an attack source and uses unicast Reverse Path Forwarding, or uRPF, to reject packets arriving from that source.

RFC 5635 describes the technique using loose-mode uRPF. In loose mode, the router checks whether a route to the packet's source exists. If the route for that source has been deliberately pointed toward a discard next hop, the uRPF check can fail and the packet can be dropped at the network edge.

In theory, that is substantially more attractive: remove the attacker without removing the victim.

In practice, it is much less generally useful.

The network performing the filtering has to deploy the appropriate uRPF behavior on the relevant interfaces. The unwanted sources also need to be identifiable enough to install useful routes for them. An attack using enormous numbers of distributed or rapidly changing source addresses turns that into a scaling problem, while heavily spoofed traffic makes source identity a poor basis for classification in the first place.

There is also an important operational difference from ordinary customer-triggered destination RTBH. RFC 5635 recommends that operators not accept source-based RTBH announcements directly from peers or customers, instead installing them through systems within their own administrative domain.

So although source-based RTBH is worth understanding, it is not something a network should assume its transit provider exposes as a customer-controlled service.

FlowSpec: filtering policy distributed through BGP

FlowSpec is where BGP starts looking much more like a filtering control plane.

RFC 8955 defines Flow Specification rules for IPv4, with RFC 8956 extending the mechanism to IPv6. RFC 8955 replaced the original RFC 5575 specification and defines a set of match components that can describe traffic far more precisely than a destination route can.

For IPv4, a rule can match characteristics including source and destination prefixes, IP protocol, source or destination ports, ICMP type and code, TCP flags, packet length, DSCP, and fragmentation state. Multiple components can be combined, with a packet matching the FlowSpec only when it satisfies the required combination.

The action is carried separately using BGP extended communities.

That action can include discarding traffic, applying a rate limit, redirecting it, or changing its DSCP marking. Of particular interest for DDoS work, RFC 8955 defines two separate rate actions:

traffic-rate-bytes, subtype 0x06, expresses a maximum rate in bytes per second.

traffic-rate-packets, subtype 0x0c, expresses a maximum rate in packets per second.

That distinction matters.

A 100 Gbps attack at minimum packet sizes presents a very different forwarding workload from 100 Gbps of large packets. The FlowSpec standard itself is capable of expressing both bandwidth and packet-rate limits.

Whether a particular router, line card, ASIC, or provider policy implements every action is a separate question. The RFC describes the control-plane language; it does not guarantee that every forwarding platform can translate every valid action into hardware.

That is a recurring theme with FlowSpec: what BGP can express and what a particular network can enforce are not necessarily identical.

Four FlowSpec gotchas worth knowing

There are several behaviors that are easy to miss when testing FlowSpec in a lab and much harder to miss during an actual incident.

First, do not assume a rate limit is network-wide. A FlowSpec rate action is distributed to routers, which then program the relevant forwarding hardware. The standard does not define a distributed shared policer that coordinates an aggregate allowance across every ingress point in an upstream network. Cisco, for example, documents FlowSpec clients programming filtering and policing into forwarding hardware on the applicable line cards and interfaces.

If a nominal 100 Mbps policer is instantiated independently at several ingress points, the aggregate traffic reaching the destination can therefore be greater than 100 Mbps. Exact behavior is implementation-dependent, so the useful question for an upstream is not simply "do you support traffic-rate?" but "what does this rate represent across your network?"

Second, port matches and fragmentation interact in a non-obvious way. TCP and UDP ports are carried in the transport header. Non-first IP fragments do not contain that header.

RFC 8955 consequently specifies that a port component does not match a non-first fragment. A rule that appears to precisely identify UDP traffic by destination port may therefore leave non-first fragments outside the match.

FlowSpec also has a fragmentation match component, so this is solvable. It simply needs to be considered when constructing the rule.

Third, the rule limit that matters is usually in hardware. FlowSpec eventually has to become something the forwarding ASIC can execute at line rate. That means rules, policers, counters, redirects, and combinations of match fields consume finite platform resources.

Those limits vary dramatically between hardware generations and implementations. Current Cisco documentation alone shows platform-specific FlowSpec policer restrictions ranging from relatively small per-slice limits on some systems to much larger shared ASIC resource pools on others.

"How many FlowSpec rules can I send you?" is therefore not a trivial configuration question. Ask what can actually be installed in hardware, whether complex rules consume additional resources, what happens when the limit is reached, and whether the provider reserves part of that capacity for its own rules.

Fourth, a syntactically valid rule can still fail validation. FlowSpec deliberately contains route-feasibility checks intended to prevent one network from injecting arbitrary filtering policy for traffic it does not control.

The original RFC 8955 procedure tightly coupled FlowSpec origination to the originator of the best matching unicast route. RFC 9117 revised that behavior so a centralized controller inside the same administrative domain can originate FlowSpec rules without having to sit directly on the forwarding path. Inter-domain rules remain subject to stricter validation.

Those safeguards are necessary. They can also produce a frustrating operational symptom: a rule looks perfectly reasonable, BGP appears healthy, and nothing gets installed because the route fails feasibility checks.

Understanding the validation policy is part of operating FlowSpec.

The attacks these tools solve well

There is an important temptation when discussing the limits of RTBH and FlowSpec to make them sound less useful than they are.

That would also be inaccurate.

FlowSpec is very good at attacks with a stable signature that can be described using information available in the packet itself.

A reflection or amplification flood is the obvious example. If attack traffic toward a particular destination can be distinguished by protocol, source port, packet length, fragmentation state, source prefix, or some combination of those properties, a FlowSpec rule can discard or police that traffic at the upstream edge while allowing traffic outside the signature to continue normally.

That is genuine mitigation.

There is no reason to send a packet through an elaborate stateful mitigation pipeline when a deterministic stateless rule can prove that it should be dropped.

Similarly, RTBH is excellent when the objective is protecting the network rather than keeping the individual target reachable. If an address is expendable during an incident and its traffic is congesting shared infrastructure, upstream blackholing is one of the simplest and most reliable tools available.

Use the narrow tool when the problem fits the narrow tool.

Where FlowSpec stops

The limit appears when the question being asked about a packet depends on packets that arrived before it.

FlowSpec can match TCP flags.

It cannot remember that a SYN previously arrived.

It can match UDP destination port 25565.

It cannot determine that a source successfully completed whatever application-level exchange your service expects before sending the current packet.

It can match prefixes, ports, protocol numbers, lengths, flags, DSCP values, and fragmentation properties. What it does not have is session state.

That becomes important as an attack gets closer to legitimate traffic.

If malicious traffic and customer traffic have the same stateless characteristics, a FlowSpec rule eventually has only blunt choices available. Drop both. Rate-limit both. Or allow both.

This is the same false-positive problem encountered by every filtering system, except here the boundary is particularly visible because the filtering language itself is stateless.

A rule cannot express "permit packets belonging to clients that completed a valid handshake during the previous five seconds" because no field in an individual packet contains that fact.

You need something maintaining state to answer that question.

There is one other boundary that is physical rather than logical: filtering only helps if it happens before the bottleneck.

A perfect FlowSpec rule installed exclusively on your own router cannot rescue a saturated transit circuit. Customer-triggered FlowSpec is useful precisely when the upstream accepts the rule and deploys it far enough toward its ingress edge that the unwanted traffic is discarded before it reaches the constrained link.

Location matters as much as classification.

What to ask your upstream

Before buying a separate mitigation product, it is worth finding out exactly what your existing providers expose. The useful questions are:

  • Do you support customer-triggered RTBH, and do you recognize the well-known 65535:666 BLACKHOLE community?
  • What IPv4 and IPv6 prefix lengths will you accept for blackholing?
  • Where in your network is a blackhole enforced, and can it be propagated selectively to peers, transit providers, or Internet exchanges?
  • What is the typical convergence time after announcing and withdrawing a blackhole route?
  • Is there any minimum duration or withdrawal hold-down?
  • Do you accept customer-originated IPv4 and IPv6 FlowSpec?
  • Which FlowSpec match components are supported in forwarding hardware?
  • Which actions are supported?
  • Do you support both traffic-rate-bytes and traffic-rate-packets?
  • When a rate action is distributed to multiple routers, is the configured value enforced independently at each point or as an aggregate limit?
  • How many rules and policers can a customer install, and does rule complexity affect that scale?
  • What happens when the hardware limit is reached?
  • What validation requirements must a customer FlowSpec route satisfy?
  • How quickly are FlowSpec advertisements and withdrawals programmed into the forwarding plane?
  • Do you offer any form of source-based RTBH, or is it restricted to your internal mitigation systems?

The answers are often more useful than a generic claim that a carrier "supports DDoS mitigation."

They tell you what traffic the carrier can actually remove, where it can remove it, and how much control you have during an incident.

Use the simple tool until the problem stops being simple

RTBH and FlowSpec are not obsolete alternatives to modern DDoS mitigation. They are components of it.

For known-signature volumetric attacks, FlowSpec can be an extremely efficient answer. For an attack where preserving the victim matters less than preserving the network around it, RTBH can be exactly the right operational response.

Their boundary appears when classification requires context: connection state, protocol state, behavioral history, or other information that cannot be inferred from the current packet alone.

That is where Wirescope's filtering begins rather than where BGP filtering ends. The same BGP communities and FlowSpec controls are handed to customers on our own transit, because the narrow tool is often the right one.

And it is also why raw capacity by itself tells you relatively little about how a mitigation network behaves under load. We cover that distinction in A Terabit Is Not a Terabit.