Prior to watching this video, my rule of thumb was anything over 8 bytes or so should not be passed by value to avoid making expensive copies. But Chandler Carruth points out that passing by const reference, or even rvalue reference can harm optimization because of aliasing. In light of that, what's a better rule of thumb for when to pass by value?
It is too clear and so it is hard to see.
A dunce once searched for fire with a lighted lantern.
Had he known what fire was,
He could have cooked his rice much sooner.
personally, I'd say that anything bigger than the width of the machine's general purpose registers should be passed by pointer or reference, but it really depends on the specific scenario. in terms of optimization, the compiler can do just about anything that doesn't change the perceived operation of the program. a reference is literally just an alias. if the object being referenced is not a built-in type, then it's likely being handled internally as a pointer anyway, so passing around a user-defined object by reference is not likely to be any slower than accessing it within the scope in which it was created. the called function may be inlined, making it effectively just a continuation of the code that called it.
in any case, you really shouldn't worry about performance and optimization, unless you determine that there really is a bottleneck after proper use of a debugger and a profiler. passing objects by reference or pointer will almost always be faster than passing by value, and with very few exceptions, never slower.
What can this strange device be?
When I touch it, it gives forth a sound
It's got wires that vibrate and give music
What can this thing be that I found?
O_o
If the type is not native, something provided by the compiler/processor, size has never been a good rule of thumb. The cost of generating a copy depends on the complexity of the object which can be anything unrelated to the size. (My variation of `std::vector' is exactly 8 bytes on 32 bit pointer platforms.)
I didn't watch the video (I tried, but the audio was killing me.), but I except some collusion of several issues related to aliasing, reference parameters, pointers, and move semantics. I'm not trying to say you are worrying about nothing, and certainly, I may be the one wrong, but it is my experience that `const &' isn't an aliasing problem for optimizers until other references of different types or pointers to family types are involved.
If you only have reference, any flavor, parameter, a reference shouldn't be a problem as an optimizer has no potentially overlapping construct to worry about. (I reference this specifically because it comes up a lot with operator overloading and meta-programming.)
If you have multiple reference, any flavor, parameters referencing only different types, types unrelated by family, many optimizers will often pretend they can't overlap because the same memory can't really be two different things. (If I recall correctly, this was explicitly codified in C99 and C++11.) Actually, this has been an unexpected issue for some; some compilers have offered options to turn off this assumption because sometimes people write unusual code.
Rather than find a good rule of thumb for using reference parameters versus value parameters, just reach for reference parameters, for objects that aren't native, while considering changes that would be necessary to prevent problems arising from aliasing.
Examples:
Use a `restrict' macro: define a macro, say `CXXRESTRICT', so you can start using specific `restrict' forms for different compilers until such time as `restrict' finally hits C++ proper.
Consider the cost and complexity of code which is neutral to exceptions: code that doesn't care about exceptions, in that the code always guarantees commit/rollback semantics exceptions or not, is a "best practice" where many canonical forms totally prevents any problem related to aliasing because only one access, a mutation, is made to relevant object.Code:void Go ( /**/ , const SObject CXXRESTRICT * fObject ) { // A compiler that honors some form of `restrict' will // produce code that doesn't care if the relevant memory areas // overlap allowing for more aggressive optimizations. }
Consider the potential availability of reference counting and move semantics, especially if on C++11: if code doesn't need read/write access to a parameter, consider making it a return value instead of a reference parameter as a compiler with "RVO"/"NRVO" will often produce great code because the compiler is responsible for managing the memory so knows the memory isn't aliased.Code:void Go ( /**/ , SObject & fOther ) { using std::swap; // We do this just in case a proper implementation does not exist. SObject sCopy(fOther); // Here we use `sCopy' many times. swap(fOther, sCopy); // We assume `swap' has a proper implementation. }
*shrug*Code:SObject Go ( /**/ ) { SObject sResult; // ... return(sResult); }
Again, I may have misunderstood. I didn't watch the video. Maybe aliasing, indirection, missing move semantics, and much complexity was part of the equation, but even then, without knowing the complexity of copying an object, I'd still say for the rule of thumb: if an object isn't native, prefer `const &' over value parameters. I know you will not see much benefit from optimizing related to potentially breaking aliasing if your object manages some expensive resource, and really, I'd be surprised if optimizers in the wild didn't treat most every `const &' as being unique in cases where a pointer to the same family isn't also involved. Of course, full disclosure, I've never looked; maybe I'm expecting too much from the optimizer or maybe a rule exist about never making that assumption in the first place.
Soma
If performance didn't matter, then there's no point in passing large object by reference. So that's a non-starter.
Access via references and pointers are potentially slow, because pointers and references cannot in general be proven not to point to the same object, so reordering access through pointers cannot be done as freely. In contrast, access to arguments passed by value can be reordered freely, because they cannot alias with anything.
But it is true that copying when passing by value can be expensive, thus my question.
It is too clear and so it is hard to see.
A dunce once searched for fire with a lighted lantern.
Had he known what fire was,
He could have cooked his rice much sooner.
O_oa reference is literally just an alias.
That is, indeed, the issue.
Your optimizer needs to behave itself in pushing this code to the assembler; if it does not, we can easily get unexpected results when these are aliases.Code:void Go ( SObject * fDOrigin , SObject * fDTerminus , SObject * fSOrigin , SObject * fSTerminus ) { while((fDOrigin < fDTerminus) && (fSOrigin < fSTerminus)) { // ... *fDOrigin = *fSOrigin; // ... ++fDOrigin; ++fSOrigin; } }
This example has the same problem.Code:void Go ( SObject * fOrigin , SObject * fTerminus , const SObject & fSource ) { while(fDOrigin < fDTerminus) { // ... *fDOrigin = fSource; // ... ++fDOrigin; } }
These are, obviously, crap examples, but the point is, the issues of aliasing come up with `const &' precisely because it would just be an alias.
Soma
Do whatever makes logical sense for your code to you as a programmer, then optimize if a profiler tells you that's too slow.
Programming and other random guff: cat /dev/thoughts > blogspot.com (previously prognix.blogspot.com)
~~~
"The largest-scale pattern in the history of Unix is this: when and where Unix has adhered most closely to open-source practices, it has prospered. Attempts to proprietarize it have invariably resulted in stagnation and decline."
Eric Raymond, The Art of Unix Programming
I did not mean that the size of internal buffers should be ignored. But plainly small objects are faster to pass by value.Originally Posted by phantomotap
Chandler Carruth is a compiler writer. He works on the clang/llvm optimizer. So when, in this video, he says pass by value is faster, I believe him. Problem is, he doesn't address when the cost of copying may be more than the gains from not-aliasing. In short, I'm sure what he says has merit, but he doesn't explain the benefits well enough, so I'm asking here.I didn't watch the video (I tried, but the audio was killing me.), but I except some collusion of several issues related to aliasing, reference parameters, pointers, and move semantics. I'm not trying to say you are worrying about nothing, and certainly, I may be the one wrong, but it is my experience that `const &' isn't an aliasing problem for optimizers until other references of different types or pointers to family types are involved.
If you only have reference, any flavor, parameter, a reference shouldn't be a problem as an optimizer has no potentially overlapping construct to worry about. (I reference this specifically because it comes up a lot with operator overloading and meta-programming.)
If you have multiple reference, any flavor, parameters referencing only different types, types unrelated by family, many optimizers will often pretend they can't overlap because the same memory can't really be two different things. (If I recall correctly, this was explicitly codified in C99 and C++11.) Actually, this has been an unexpected issue for some; some compilers have offered options to turn off this assumption because sometimes people write unusual code.
It's true that problems with aliasing can only come into play once two pointers or global variables are involved, but with functions and methods with multiple arguments this can easily happen, especially because there is no way to pass "this" by value.
Last edited by King Mir; 06-26-2013 at 03:29 PM.
It is too clear and so it is hard to see.
A dunce once searched for fire with a lighted lantern.
Had he known what fire was,
He could have cooked his rice much sooner.
That doesn't help at all, because neither passing by value nor by const reference makes any logical difference. The only difference is performance.
It is too clear and so it is hard to see.
A dunce once searched for fire with a lighted lantern.
Had he known what fire was,
He could have cooked his rice much sooner.
O_oSo when, in this video, he says pass by value is faster, I believe him.
Well from what you've said, I do not believe him. I do not believe such broad statements without context, nor would I believe such a statement with context if he doesn't address the issues I've addressed. A keynote called "Optimizing the Emergent Structures of C++" without addressing the ways in which "best practices" change the face of code optimizers actually see makes him a crap speaker regardless of his job. The "emergent structures" he is talking about, I assume having not seen the video, depend on techniques found in abstract composition. A compiler playing safe with potential aliasing will have a cost, but copying a non-native object across every call in a "functoid" designed for abstract composition is guaranteed to eventually be more expensive. The very nature of designing for "emergent structures" means designing so that a "yield" (the value of a function called) can fit anywhere, and that very idea, every "yield" potentially being the result of any number of "yield", means that each component must be fit for optimization. You can't do that, without move semantics, by preferring "pass by value".
This seems perfectly reasonable, but without getting a lot more clever copies WILL eat us alive because the `SPlus' object may be infinitely chained leading to more and more costly composition. We can solve this with move semantics, and as I said, move semantics may have been part of the discussion, but without move semantics we need to use a better strategy than "? by value" everywhere even if it works in the simple case because it will quickly grow complex.Code:template < typename FValue , typename FYield > struct SPlus { SPlus ( const FValue & fValue , const FYield & fYield ): mValue(fValue) , mYield(fYield) { } // ... FValue operator * () { return(mValue + *mYield); } // ... FValue mValue; FYield mYield; }; // ... template < typename FValue , typename FYield > SPlus<FValue, FYield> operator + ( FValue fValue , FYield fYield ) { // ... return(SPlus<FValue, FYield>(fValue, FYield)); }
Sure, this sort of thing is an extreme example, but applies to any design where "emergent structures" is the flavor,
I'm not even saying "pass by value" can't be faster than "pass by reference"; certainly, I could show infinitely many example where that is the case, but without discussing "pass by value is faster" in context, the statement is entirely meaningless.
A "rule of thumb" is inherently a guess; if you really want to wager on "pass by value" being faster, go for it.
Soma
As best I can tell, a more fitting title for the talk would be "What code is hard to optimize by a modern compiler". The answer being, code that has memory shared between functions.
It is too clear and so it is hard to see.
A dunce once searched for fire with a lighted lantern.
Had he known what fire was,
He could have cooked his rice much sooner.
Memory shared between functions is handled by "global" optimization (within a single source file). I'm not sure which optimizations would be most difficult for a compiler, for example another challenging optimization is trying to optimally place the most frequently used native variable types in registers.
O_oAs best I can tell, a more fitting title for the talk would be "What code is hard to optimize by a modern compiler".
I think "most" would have been simpler.
Seriously though, I was under the impression the talk was about implementation strategies which give the compiler a better chance to optimize. In other words, despite knowing who he was, I thought the talk was about giving preferred tools (in the brainbox form) to average C++ programmers. If the talk was framed from the opposite direction, categorizing problematic constructs, the lack of context for solutions is less damning simply because there would be too many cases to cover in a keynote presentation.
With that framing in mind, I agree with making programmers aware of the aliasing problems and what optimizers must avoid without regard for alternatives to copying because aliasing problems come up way more often from avenues other than `const &' function parameters.
I know none of this helps you find a "better rule of thumb", but I'm not sure one exists. You say you believe him, and in the general sense of warning programmers off needless aliasing I too believe him, but that doesn't really change anything for me. I already write a lot of code with the most significant exception guarantees; obviously, a lot of my code doesn't have aliasing problems because I'm already using a copy. I also make prolific, almost religious, use of the iterator patten in multiple stages which forbids "pass by value" by nature. I may be losing something to aliasing prohibiting a certain optimization, but then, lower level stages mostly operate on a single object.
As I said earlier, instead of trying to find a "better rule of thumb", why not continue to reach for `const &' parameters so that you can put your effort into finding ways that limit the costs of rigid code in the face of aliasing? Look at my examples again, if you do nothing different, but pass by value obviously, you may pay the price of a copy which may be more of a cost than aliasing. Instead of just making that change, what other changes could be made in the process to offset the cost of a copy in other ways? If you can rewrite a facility in such a way that having a copy provides a significant benefit, the cost of the copy isn't relevant so you can reach for a "pass by copy". (You'd still be defaulting to "pass by reference" in the general case.) I don't mean to harp on exception safety, but again, I consider an object being in pristine condition after an exception is raised while attempting to mutate an object to be well worth the cost of a copy; the copy never costs me anything, so I avoid the cost of potential aliasing in any event.
Soma
I'm going to guess that the video mentions examples such as:This clearly presents an aliasing issue, where passing by value would not.Code:#include <iostream> int foo = 0; int foobar(const int &bar) { foo = 42; std::cout << bar; } int main() { foobar(foo); return 0; }
I don't know of a better rule, nor do I see a reason to change.
My homepage
Advice: Take only as directed - If symptoms persist, please see your debugger
Linus Torvalds: "But it clearly is the only right way. The fact that everybody else does it some other way only means that they are wrong"
I suspect you're using a slightly different definition of alias than I am. what I meant was that, from the programmer's point of view, a reference is just another name for the same object - an alias. no implied pointer magic in my statement. I realize that the word alias can take on a slightly different meaning in C and C++, but that's not what I was talking about.
What can this strange device be?
When I touch it, it gives forth a sound
It's got wires that vibrate and give music
What can this thing be that I found?
Well, surely that would imply that as far as you're concerned it doesn't really matter, and you should just do either?
For me, I find the distinction helpful when reading a function - using a const reference says both (i) I don't need a fresh copy of that object to work on and (ii) I don't need to change that object in order to do whatever it is I'm doing. But if you don't think like that, then the general advice from before becomes "in this instance just do either one, then optimize if a profiler tells you it's too slow".
Programming and other random guff: cat /dev/thoughts > blogspot.com (previously prognix.blogspot.com)
~~~
"The largest-scale pattern in the history of Unix is this: when and where Unix has adhered most closely to open-source practices, it has prospered. Attempts to proprietarize it have invariably resulted in stagnation and decline."
Eric Raymond, The Art of Unix Programming
