Yeah, everyone struggles with that, especially since AWS doesn’t really break down costs in a way that makes sense if you’re trying to work out which business unit or feature is costing you money (unless you set things up where every team has their own account or whatever, and even then).
I think Netflix could probably start selling their tool and make more money from that then streaming if they can get it to work, because I’m guessing AWS makes it hard on purpose.
There’s a lot of overlap of features. For example, we have a serverless function that does one thing for a dozen different departments, as well as for affiliates. Right now, our solution is just to say its part of the “operating expenses” bucket, which is our black box of things that just need to be paid.
The solution to fix this would be to split the function as separate instances, to get cleaner tag data.
But now you have separate instances of the same code and everything is even less efficient/effective, and significantly more overhead for “cleaner” tagging.
The serverless function is a extreme tiny example. Multiply that by bigger projects/critical architecture/etc.
For our company, we accept not really knowing the exact details, because the cost of getting that clarity is a magnitude more work for little gain.
My take is that it’s already your systems feature, rather than admins responsibility. If you treat departments like customers, you’d find a good way to spread the costs. If something is just a „common infrastructure”, you will always find something that makes costs that doesn’t have an easy way to track who triggered that - because you don’t pass enough information with it.
That doesn’t work in all cases. I’ve recently come across two examples where we had a hard time explaining our costs even though we extensively tag and even have fine-grained AWS accounts:
Some costs can’t be tagged or at least not easily, e.g. custom CloudWatch metrics.
For some resources it makes a lot more sense to provide them centrally for multiple services at once, e.g. NAT gateways or load balancers.
Not only is that free, but I can’t imagine a better alternative. And they would have the same issue with allocation on prem. WithOUT tagging and Cost Explorer.
Yeah, the actual headline to the article is “Even Netflix struggles to identify and understand the cost of its AWS estate,” which OP has very unhelpfully shortened to post “Netflix struggles to understand its cloud costs.”
The word “Even” is doing a lot of work, and leaving it out changes the meaning of the headline.
Of course. But even for people who don’t read the article, it’s still best practice to just copy the headline from the article so that it’s the top of the Lemmy thread.
I think Netflix could probably start selling their tool and make more money from that then streaming if they can get it to work, because I’m guessing AWS makes it hard on purpose.
The company Cloudhealth Technologies has already been doing this for years, and not just with AWS but other cloud providers as well. Unfortunately they’ve been acquired by VMware so no idea if they’re still as good as they used to be…
This just isn’t true. AWS is very clear about pricing and provides pretty good tooling for analysis. Complex infrastructure will have complex costs though. AWS has its problems a’plenty, but unclear costs isn’t one in my opinion.
I agree, works fine for my stuff, but I’m pretty heavy on vanilla ec2, RDS, ec.
I can see some people running some of the more complicated products and getting flustered. Some of the compound stuff where you’re paying extra transaction costs on top of the serverless things
Yeah, everyone struggles with that, especially since AWS doesn’t really break down costs in a way that makes sense if you’re trying to work out which business unit or feature is costing you money (unless you set things up where every team has their own account or whatever, and even then).
I think Netflix could probably start selling their tool and make more money from that then streaming if they can get it to work, because I’m guessing AWS makes it hard on purpose.
Not to mention the costs change on the fly since load changes on the fly.
And yea AWS is like a used car dealer, they want the TCOS to be hard to calculate … “Oh you want the car to have wheels. That’s an extra fee…”.
Not sure what is hard in it - you need consistent tagging, and that by itself gives you a lot of mileage in cost explorer.
There’s a lot of overlap of features. For example, we have a serverless function that does one thing for a dozen different departments, as well as for affiliates. Right now, our solution is just to say its part of the “operating expenses” bucket, which is our black box of things that just need to be paid.
The solution to fix this would be to split the function as separate instances, to get cleaner tag data.
But now you have separate instances of the same code and everything is even less efficient/effective, and significantly more overhead for “cleaner” tagging.
The serverless function is a extreme tiny example. Multiply that by bigger projects/critical architecture/etc.
For our company, we accept not really knowing the exact details, because the cost of getting that clarity is a magnitude more work for little gain.
My take is that it’s already your systems feature, rather than admins responsibility. If you treat departments like customers, you’d find a good way to spread the costs. If something is just a „common infrastructure”, you will always find something that makes costs that doesn’t have an easy way to track who triggered that - because you don’t pass enough information with it.
That doesn’t work in all cases. I’ve recently come across two examples where we had a hard time explaining our costs even though we extensively tag and even have fine-grained AWS accounts:
Not only is that free, but I can’t imagine a better alternative. And they would have the same issue with allocation on prem. WithOUT tagging and Cost Explorer.
That’s the point of the article
Yeah, the actual headline to the article is “Even Netflix struggles to identify and understand the cost of its AWS estate,” which OP has very unhelpfully shortened to post “Netflix struggles to understand its cloud costs.”
The word “Even” is doing a lot of work, and leaving it out changes the meaning of the headline.
Also, other than the headline, you know, the article itself :)
Of course. But even for people who don’t read the article, it’s still best practice to just copy the headline from the article so that it’s the top of the Lemmy thread.
The company Cloudhealth Technologies has already been doing this for years, and not just with AWS but other cloud providers as well. Unfortunately they’ve been acquired by VMware so no idea if they’re still as good as they used to be…
This just isn’t true. AWS is very clear about pricing and provides pretty good tooling for analysis. Complex infrastructure will have complex costs though. AWS has its problems a’plenty, but unclear costs isn’t one in my opinion.
So Netflix built this tool just for fun then? Obviously they feel AWS’ tools are lacking.
deleted by creator
I agree, works fine for my stuff, but I’m pretty heavy on vanilla ec2, RDS, ec.
I can see some people running some of the more complicated products and getting flustered. Some of the compound stuff where you’re paying extra transaction costs on top of the serverless things