Other restrictions besides bandwidth exist. For example, DPI requires more CPU and memory, which are some of the major constraints of networking equipment, especially when the equipment resides at the edge (borders) of a Tier 1 ISP.
from (http://en.wikipedia.org/wiki/Narus_(company)) we have the following "A single NarusInsight machine can monitor traffic equal to the maximum capacity (10 Gbit/s) of around 39,000 DSL lines or 195,000 telephone modems. But, in practical terms, since individual internet connections are not continually filled to capacity, the 10 Gbit/s capacity of one NarusInsight installation enables it to monitor the combined traffic of several million broadband users."
I think you're underestimating the limit to how fast DPI is or can get. The computations aren't complicated and like many kinds of algorithms you can trade space with time.
This statement might be correct (although I've never seen any evidence to support it), but it's still misleading because "forward the packet" is always going to require fewer resources than "read the packet, parse it using this set of algos, use the parse results to search your DB of shit you want to fuck with, optionally fuck with the packet, optionally forward the packet". An exception to this would be if you have big iron on the edge that protects resource-poor interior nodes. This situation is unavoidable sometimes (DDOS), but it's not what anyone should aim for.
They don't necessarily need to do anything with the packet then -- they could always pass a copy to cold storage, and then crunch the bits at their leisure in a massive data center.
A) this is probably true to an extent.
B) slurping everything to disk is impractical, and no one has that much storage, so you're back to parsing and deciding at the boundary. This "leisure" time doesn't ever happen when you save everything all the time.
C) this point seems to contradict the various claims of reasonable network maintenance I've seen; if it's trash you want to drop, you want to drop it on the floor not on your disk. If you're keeping actual traffic rather than just summaries of traffic, you're not doing reasonable network maintenance.
I think you're underestimating how much traffic can be out there.
DPI requires decoding application-layer protocols. This requires many layers of decoding. That's all well and good for one client (feature phones happily load web pages, after all), but to do it inline at the router means the router needs to be as powerful as the sum of all of the networking hardware of the hosts it routes to.
Source: at my last job, we were working on fixes for security holes that could cost billions of dollars of damages per incident, and the fixes were solvable in software, but we couldn't persuade customers of the value of buying beefy enough network equipment to actual block the attacks. Mind you, I didn't run any numbers myself on the cost of the hardware; that's just what my colleagues said.
Oh, it's obviously possible. Reading a little about Narus does not give me the impression that it's cost-effective, for any non-NSA sense of the word "cost". In any case, in my opinion our customers were not taking their problems very seriously, and their price sensitivity reflected that.
If you have any trustworthy source of ballpark pricing for a 10Gb/s DPI solution, Narus or otherwise, I'm curious. I'm talking BALLPARK here.
I looked up some of Narus's whitepapers, and what I'm seeing looks like Grade A Bullshit. That doesn't mean they ARE bullshit -- their whitepapers would look like this in either case. But I would like to point out that it is absolutely possible to build a company like this without actually delivering the goods, because many of your potential customers are largely incapable of understanding whether or not the goods have really been delivered.
ALSO, I guess I should remember that DPI for stopping malware, and DPI for spying on citizens, are really different things. When you're spying on citizens, 90% effectiveness is good enough. When you're trying to block malware (versus e.g. commited cyberterrorism), 90% is a little better than 0%, but not much, because your adversary can try many times and they only have to succeed once. So in that sense, the DPI being discussed here is a much easier problem than the DPI that we were trying to tackle.
well, i think you can look up the pricing for gateway nodes for cellular wireless equipment vendors (hspa/umts/lte/cdma etc.). my guess is that it should somewhere around 200-400k range. a sufficiently capable box can handle approx. 7 - 10m subscribers simultaneously doing data.
any cellular network that you send data on, already does this. standardization only levels the playing field.
Bandwidth is the measure of data over time. DPI is a process which inspects each piece of data and that takes time. Therefore, the bandwidth will decrease.
Latency != bandwidth. Also, one goal of DPI is to enable granular QoS policy. So the total bandwidth available would stay the same, but the bandwidth available to each app would vary depending on network conditions and the will of the network operator.
Wikipedia says, "In computer networking and computer science, the words bandwidth, network bandwidth, data bandwidth, or digital bandwidth are terms used to refer to various bit-rate measures, representing the available or consumed data communication resources expressed in bits per second or multiples of it (bit/s, kbit/s, Mbit/s, Gbit/s, etc.)."
All else being equal, increasing the denominator reduces the rate. It's just math.
And it's the wrong math. You claim that increasing latency will necessarily decrease bandwidth, and that is simply not the case.
If we assume that there are a fixed number of packets in flight at any given time, and we increase the flight time of all packets, then it will be true that the packets-processed-per-time will drop (and ceteris paribus that the bandwidth would drop). This assumption, however, is incorrect. One could (in theory, and within limits in practice) add extra pipeline steps to do extra processing, increasing the time taken to process each packet, without decreasing the rate at which packets traverse the processor (e.g. the router). This is hopefully what happens.
Of course, as I and others said elsewhere, it is OFTEN the case that as processing-per-packet increases, this becomes a bottleneck on packets-per-time, and this does eventually limit the bandwidth. (Jumbo frames sometimes help, but not if the processing is being done on the entire application payload, which of course is what DPI does, so jumbo frames ain't gonna help there.) So in practice, more DPI might well decrease bandwidth.
If you would like an analogy, imagine the hosts are cities, the packets are trucks carrying goods, and the route is a highway of fixed speed limit and width. (Yes, one weird thing about this analogy is that building new trucks is basically free and done on-demand.) Suppose we consider shipping an infinite amount of coal from city A to city B. The number of tons of coal shipped per day does not depend on the length of the highway. The amount of time it takes for each ton of coal to get there DOES depend on the length of the highway. Coal-per-day is like bandwidth, hours-per-truck-trip is latency. You can increase or decrease one without affecting the other, all else being equal.
Note: really, I shoulda done that example as a series of tubes, oh well.
You're right, I said something silly and invalid. What I was trying to say was simply that an in-series DPI implementation will increase round trip times and decrease bandwidth.
My personal understanding of DPI is that they want to do it in-series so as to be able to reset TCP connections immediately upon detecting a hit, but that might not be how they do it.
"In TCP connections, the large bandwidth-delay product of high latency connections, combined with relatively small TCP window sizes on many devices, effectively causes the throughput of a high latency connection to drop sharply with latency" --Wikipedia
It does equal bandwidth until everything is jumbo frames UDP
If you would have pasted a larger part of the quote, you would have included the part of increasing the window size (e.g. window scaling, selective ACKs, like almost everything out there supports) and the mention of satellite links to refer to high latency conditions. And that notwithstanding that, we are talking about additional latency of microseconds against a typical delay of at least 30 milliseconds. "drop sharply" just doesn't apply here.
If you want to argue against DPI that's fine but "it'll make the Internet slower" comes off as whining plus you're fighting against the exponential effects of Moore's law. A quick Google search tells me there are several products that'll do line rate DPI at 10Gbps.
There are much better arguments to be made against DPI such as privacy, a slippery slope to a walled garden, or just plain unfairness.
The main limit to bandwith is speed of routing the packets. This is why QoS typically fails for IP; it is usually cheaper to just use a faster dumb router than to use a slower smart router.
