Backpressure & Resilience

What is backpressure?

In the software world, counterpressure is an analogy that takes over the flow dynamics of car exhaust gases and home installations. In this analogy, the desired fluid flow of the pressure leads to a desired data flow and in the case of a house installation to a desire for a flow flow. The purpose of the software is to capture the input data and transform it into the desired output data, and vice versa. A counter-pressure occurs when there is too much input pressure to convert the input into output. A resistor is a resistor with an output that is calculated as fast as the inputs arrive. The easiest way to view resistors is by far the most common way to display pixels on a monitor: with a computer monitor. By the way, you may have heard someone use the word 'counter-pressure' to actually mean something that has the ability to control and handle counter-pressure. I think a lot of people had their "aha moment" after hearing this example, but what counterpressure really means can be a bit of a blur. Other forms of counterpressure occur when the software waits for the user to do something, such as input to the computer monitor or output from the monitor.

Back Pressure Examples

Now we have talked # about counterpressure from software, but the most common case is when it comes to file systems. Writing to a file is slower than reading from it, and vice versa, so writing to files is slower than reading from them.

Now imagine that you are doing this with a 6GB file, and when you have read the file completely, you have a 2GB buffer to write to. Now there is a growing deficit, but you will not be able to catch up and repay the debt until you have read all the input files in full.

Imagine that this is a lot of wasted memory, and on some systems it may even exceed the amount of available memory. This is because the web server performs operations on multiple requests simultaneously, so imagine there is a lot of memory waste.

I hope it is clear that this approach is not practical, but don't worry, the solution is simple: just read as fast as you can write. This library automatically provides you with abstractions, often with the concept of the current pipe.

Node - jsbe a great example, but it is also available in many other languages, such as Ruby, Python, Ruby on Rails, Java, JavaScript, PHP and even Java EE.

Nowadays, it is very common to use micro-service architectures where multiple servers share responsibility. Back pressure is created when a server sends requests faster than it can, for example, handle more than 100 requests per second. The next example is the communication between servers: if server A sends 100 rps requests per second to server B and serverB can only process 75 rPS, then they have a deficit of 25 rps.

When server B has to process something or communicate with other downstream servers, it falls behind and has to somehow cope with the counterpressure.

If the increase remains constant, the Rp-25 deficit will soon fall and disappear from memory, so one of the options is to buffer it.

Z could instruct service A to control the vendor and slow it down by buffering the request until it eventually runs out of memory, but even that is not always practical. Sometimes you cannot control the user or even tell him to slow down, so deleting requests is not acceptable. However, it is often better to have all requests to the server buffer in order to better allocate the memory load and not affect other requests. It is a fairly common request that deleting a request is an option, and sometimes this can cause one server to send another request.

Failures may be inevitable in this case, so limit the scope and prevent a cascade of denial of service. If a non-functioning service is not allowed to deny the same access as the other, the other two services (B and C) must be maintained, and this prevents cascades.

That's the crucial part, and when I worked at Netflix, that kind of counterpressure was common. In one of my lectures, I talked about a scenario I talked about in the movie "The Hunger Games": Someone buffered, but there would be a producer who would then form a buffer to control his own producer who could not control him.

If you are interested in handling services on a scale, you want to learn to test resilience, failures, etc. Sometimes there are strategies to control back pressure, but ultimately it depends on whether the manufacturer controls things like RSocket and gRPC.

This could be an attempt to render a huge list, deflect a fast keyboard event, or display a WebSocket that sends 20,000 events per second. The final example of counterpressure is rendering an interface in an app, but it cannot be rendered at the same time as the rest of the app. This counterpressure occurs when the user interface is displayed in the context of a large number of events (e.g. keyboard events). I'm not the only one. Focusing on the example of Web Socket, because this kind of counterpressure can be very complex and often involves "death by a thousand cuts," is therefore easy to see how this can go wrong.

If a WebSocket sends 20k, 100k or even messages per second, you may not be able to render all these messages simultaneously with the rest of the app (e.g. the browser).

So some kind of counter-pressure strategy is required, such as a multi-threading strategy or even a client-side counter-pressure strategy for the browser.

If you cannot control the rate on the server, you need to create a client-side buffer at that rate. In case of buffering, you can collect all incoming messages in an array or render list, request an animation box and render the list in the DOM. If you are appending to an existing table, you may want to use a form of table virtualization, because rendering 100k rows will also be a huge bottleneck in the form of backpressure.

Depending on the actual number of incidents, one approach can work and the other is filtered, otherwise the only other option is to drop it. Technically, there's no need to ignore the counter-pressure, which, to be honest, isn't a bad idea if you're like me.

Introducing more complexity also comes at a price: you don't have to be lossy and store data, and you don't end up using excess memory buffers. Although this is perfectly possible, manufacturers' controls do have their share of problems, but by far the best option is manufacturers' control.

Unfortunately, manufacturers are not always able to verify this, and it can cause a lot of problems, especially in the case of high-end devices.

This is a clear case when it comes to user input, but it is not easy for the user to control it. Try it yourself, and if it doesn't work, you can try it again and again with another device.

However, always remember that if your buffer is unlimited, it is dangerous, as this means that you have no size or time limit for it. Unlimited buffers are a common cause of memory crashes on servers, so always ask yourself if it is possible that the rate at which the buffer grows will ever exceed a rate that drains it for a significant amount of time. If that is not the case, then it will be forever that most people will reach for next and so on.

In fact, it has been argued that buffers should never be unlimited, and an example of this can be found in the server communication example above.

I would say that it is often better to start dropping than to fall completely out of memory, but I am not so strict. Dropping is the last strategy, and as already mentioned, it is often combined with a buffer, so I would not transfer the sample in the same way as in the previous example.

User Experience (UX) can often help you decide which approach to take, and it is a good idea to update the table every 100 000 seconds, even if you could. Would your users prefer to see the example updated every second, or should you redesign things to send a stream so they can read the database more slowly if needed?

It is very common that developers spend a lot of time optimizing performance and end up with a bad UX, while a good UX has no performance problems from the start. Only unique circumstances can bring light into the darkness, so remember that UX can guide you, but only when it is possible. I want to keep most of these posts independent of the language or platform in which you are programming, as counterpressure is a problem you have to deal with.

This is usually a one-to-one request - response style, but there are many other types, such as a pull-based stream where the producer is controlled by the consumer. Unfortunately, the term 'electricity' is ambiguous, and there is also a lot of confusion about the difference between a consumer and a producer, and between the two types of application.

Depending on who you are talking to, this can be considered a hybrid push strategy - pull strategy when you press an answer asynchronously, and a normal synchronous iterator is considered a traditional pull current. Others do not differentiate between asynchronous and synchronous, but only call it pull. For the JS people, it is also a port to other languages, or, depending on the talk, you can consider it a "hybrid push / pull" strategy where pull is the only move.

A push stream is often used when it comes to entering users, but they are precisely modeled for manufacturers because you can't control the user. With push-based streams, manufacturers have the control to push the data to consumers when the data is available and then push it back to consumers.

When I first heard the term backpressure, I admitted that I was intimidated, but the truth is that it is a real thing. And if you know how to fight them, you can tackle even bigger problems by orders of magnitude. I feel like I'm using jargon to make that person sound smart, and unfortunately that's sometimes the case. Counterpressure is the difference between handling fast mouse movements and managing thousands of servers.