Thread: is the STL widely used in professional C++ programming?

You shouldn't take hyperbole literally.

Anyway, I'm pretty sure that EA's STL implementation played a very important role in the development of C++0x's scoped allocators.

Originally Posted by phantomotap

Which is a really stupid thing to say. A new or better computer may buy you a few percent performance over what is, but a new or better algorithm or implementation may buy you a few percent forever.

(A better implementation granting the same performance increase as a better machine will likely provide a similar advantage to any future machines.)

Soma

I think the point of that saying is that it's faster & cheaper to buy a faster system than to optimize it right now, just so they can finish it and start making some money sooner. Later they can go ahead and optimize it if they need to, but getting something out the door as soon as possible is usually the most important thing to companies.

Now that is ridiculous. Then every gamer who buys it has to suffer instead.

Originally Posted by cyberfish

I don't actually know much about allocators since I've never had to use them. I thought it's just providing a malloc-like and a free-like function, and when they are called is up to STL. Am I mistaken?

That is sort of correct. The allocator also provides functions for constructing objects from that memory.

The default-supplied allocators (i.e. the ones used by standard containers unless you tell them to do differently) essentially just provide dynamic allocation of raw memory. Functionally, that is equivalent to malloc() and free().

User supplied allocators obtain memory from other places (eg from shared memory, via some other API, etc). However, they provide the same style interface.

Originally Posted by cyberfish

Even if it's possible, if you were to go that far to make sure STL containers use memory exactly the way you intend it to, wouldn't it be easier to just write your own containers?

Maybe. You'd need to specify the functional requirements your container is required to meet, and the constraints in which it is required to meet them.

In particular, you need to provide an objective justification for using memory in "exactly the way you intend to". What benefits are you seeking, and what trade-offs are you accepting that make your container preferable to the STL containers?

Originally Posted by cyberfish

If you do all the memory management yourself, exactly how much work is STL doing for you still?

The STL containers do the memory management they need using the allocator. The allocator acts as the intermediary to raw memory.

The entire purpose of an allocator type is (in your words) to allow control over WHERE (e.g. from shared memory or a database) and HOW (eg. by what API) raw memory is allocated. The container type itself controls WHEN memory is requested and the QUANTITY asked for in each request.

The different types of strategies used by different containers represent different functional and non-functional trade-offs (eg allocation strategy).

Each allocator type is required to provide a defined interface by which each container may make its requests for raw memory, and for initialisation of that raw memory (i.e. construction of elements)

Originally Posted by cyberfish

I know our multi-platform engine doesn't use STL, and it's built from ground up in-house (the studio is only a few years old). Most of the people there worked a long time at EA (since we are across the street from an EA studio), so I wouldn't be surprised if it's like that at EA, too (at least at some studios, for some projects), since they are used to it.

From your comments, the message I am receiving is that your developers are either working with a "not invented here" mentality, or have simply stuck with an in-house approach due to lack of understanding of what the STL actually allows them to do.

Neither is a valid reason to exclude the STL (and, equally, not a valid reason to use it). Choosing between an established in-house approach and a standard library requires an informed comparison of the two. In other words, a list of specific functional and non-functional requirements against which both approaches may be compared.

Originally Posted by cyberfish

That's very easy to say. Even I can say it. Example?

How do you write an allocator that makes sure std::list allocates 5 blocks at once?

You wouldn't. An allocator is not the place to enforce such a constraint. That is in line with what I said above: it is the container that decides when it makes a request of the allocator and, when it does, how much it requests.

Such a restriction (allocating 5 blocks at once) would also be less efficient by various objective measures than the default strategy used by most std::list implementations, which doubles the buffer size every time a new allocation is requested. (That default behaviour gives a linear execution time - which is required by the standard - for single appending or insertion of elements into the list. I suspect allocating five blocks at once would not meet that requirement.).

How about you explain what the purpose would be in limiting a linked-list-style container (or any other container, for that matter) to allocating five blocks at a time? In particular, clarify the circumstances and measures by which such a constraint is considered most or less effective/efficient/whatever than any of the standard containers.

I'm curious to know how that container would behave if you attempted to add six (or more) elements to it.

The reason I ask is that I'm curious to know the circumstances in which I would prefer your in-house containers over the STL. Or vice versa. Assume in your response that, I'm not obligated by any policy to use either your in-house containers or the STL containers.

My concern is that I look at the "five blocks at once" as an implementation choice, and can't see what functional (or non-functional) requirement it would actually address.

Yeah, grumpy kinda confused vector and list there.

Anyway, writing an allocator that allocates list nodes in blocks of 5 isn't very hard. Writing an allocator that can free those blocks again is tricky, because it basically needs to track which of those blocks are completely freed and which are only partially freed. (But so would any list implementation that employs this strategy.) There are various ways to do that, with different tradeoffs. Generally, though, the overhead of this approach suggests that blocks sizes of 8 or higher are probably better.
Writing an allocator that makes sure that only one list ever gets nodes from the same block should be easy, but isn't, due to the broken specification of allocators in C++03. In C++0x it's going to be easy. But it can also lead to some nasty hidden inefficiencies in some code, due to swapping no longer being O(1).

That said, though, one ugly thing about the STL is that the allocator is part of the type. That makes usage annoying, because it means that every function that operates on, say, a list of ints will still have to be a template to accommodate different allocators.

Not this one, no. Scoped allocators and the adapter give you a lot more control over memory allocation in containers, making arguments about needing custom containers for custom memory management even weaker, but they don't change the fact that the allocator is part of most classes using one, in particular all the containers. The classes where this isn't so are function, promise, and packaged_task.

Originally Posted by cyberfish

So I see you've worked in the game industry, went through making this decision, understood and weighed the arguments for and against custom containers in this case, and decided STL is better.

Or are you just naively saying STL is the answer to all of life's problems in C++, so doing it any other way is wrong?

It certainly sounds like it.

No, it wasn't directed at STL. It was directed at the "if something is slow, upgrade, don't optimize.".
(The STL is something I'm not about to discuss.)

Originally Posted by cyberfish

Using the STL brings in a lot of risk.

With a bunch of hacks we can probably make it perform well on one platform. What if the publisher suddenly comes in and tell us to port it to some new platform? We do it, build, it compiles, runs, and gives us 1fps. We'll profile it, and the culprit turns out to be STL. What do we do then? Re-hack everything for this STL?

What if your custom containers give you 1fps on the new platform?

Originally Posted by cyberfish

What if a new version of Visual Studio comes out where they changed the STL a bit, and suddenly we see a 50% performance drop?

Then throw away the new STL and use the old one. They're just a bunch of header files that you can copy around...

If a company already has their own containers that they've tested and they're using, and they're happy with them, then obviously they should continue using them. If you're starting from scratch though, you should probably at least consider whether or not using the STL or writing your own is better.