MSDN volatile sample

This is a discussion on MSDN volatile sample within the C++ Programming forums, part of the General Programming Boards category; The only thing I took away from that discussion was that you don't need volatile because you can always make ...

  1. #16
    Captain Crash brewbuck's Avatar
    Join Date
    Mar 2007
    Location
    Portland, OR
    Posts
    7,239
    The only thing I took away from that discussion was that you don't need volatile because you can always make a call to a "dummy function" which forces the compiler to synchronize the memory location. While true, it seems like a dorky hack.

    Also, it seems this approach would not work with a non-global variable, for instance a variable accessed through a pointer which is shared between threads. The pointer itself might be synchronized, but the value it points to won't be, unless it is declared volatile.

    You could write thread safe code without volatile, just as you could build a house with no nails, but it's not inherently a better or worse way of doing things. Just be aware that volatile makes guarantees about the reads and writes on specific variables, not the interactions between those variables. You still have to understand barriers, etc.

  2. #17
    Registered User Codeplug's Avatar
    Join Date
    Mar 2003
    Posts
    4,669
    >> The only thing I took away from that discussion was that you don't need volatile...
    That was my conclusion as well.

    gg

  3. #18
    Cat without Hat CornedBee's Avatar
    Join Date
    Apr 2003
    Posts
    8,893
    You're always at the mercy of the implementation. C++03 doesn't have the concept of parallel execution. The memory model does not support it. The execution model does not support it. The moment you call any thread creation function, you're in implementation-defined area.
    All the buzzt!
    CornedBee

    "There is not now, nor has there ever been, nor will there ever be, any programming language in which it is the least bit difficult to write bad code."
    - Flon's Law

  4. #19
    Captain Crash brewbuck's Avatar
    Join Date
    Mar 2007
    Location
    Portland, OR
    Posts
    7,239
    Quote Originally Posted by Codeplug View Post
    >> The only thing I took away from that discussion was that you don't need volatile...
    That was my conclusion as well.

    gg
    Cutting my quote off to make me look like I'm agreeing with you is dumb.

  5. #20
    Algorithm Dissector iMalc's Avatar
    Join Date
    Dec 2005
    Location
    New Zealand
    Posts
    6,304
    Well I certainly didn't quite take away with me the understanding that the volatile keyword should never be used, after reading those. It's perhaps compiler dependent though.
    I might not go overboard and use it everywhere I think there might otherwise be a problem, but you can be damn sure that if there is an intermittent problem in my multi-threaded code that it'll be one of the things I'll try!

    Any thoughts on atomic operations and why functions like InterlockedIncrement have a volatile pointer as their parameter? Is that necessary?
    http://paste.lisp.org/display/53106/raw
    Actually the "register volatile" stuff seems a bit strange...
    Last edited by iMalc; 12-29-2007 at 12:37 PM.
    My homepage
    Advice: Take only as directed - If symptoms persist, please see your debugger

    Linus Torvalds: "But it clearly is the only right way. The fact that everybody else does it some other way only means that they are wrong"

  6. #21
    Registered User Codeplug's Avatar
    Join Date
    Mar 2003
    Posts
    4,669
    From a debate point of view, here are my assertions:
    1) This is a non-POSIX environment debate - thanks to the memory visibility guarantees given by POSIX.
    2) There is no standard, including POSIX, which gives any meaningful semantics to applying volatile to a *shared* object. (*shared* = accessed by multiple threads).
    3) Therefore, any meaningful semantics must come from the compiler (documentation).

    Why write code that's bound to "implementation defined" behavior - specially when the same code can be written without "implementation defined" behavior?

    >> Any thoughts on atomic operations and why functions like InterlockedIncrement have a volatile pointer as their parameter?
    volatile was added to the Interlocked API's sometime after the VC++ 6.0 SDK. (Found a reference that said the signatures changed in the July 2001 PSDK.)
    This has always baffled me. While searching the groups for some answers, I found this great thread in microsoft.public.msdn.general:
    http://groups.google.com/group/micro...2a440f44693ea7
    I say "great" not only because my position is backed by a couple of MSMVP's - which prob. ain't say'n much - but it also has a couple guesses as to why volatile was added to the Interlocked API's.

    There's plenty of further reading on this subject - for those who like to dig. Summaries and more links here:
    http://groups.google.com/group/comp....5048032700caf/

    gg

  7. #22
    Algorithm Dissector iMalc's Avatar
    Join Date
    Dec 2005
    Location
    New Zealand
    Posts
    6,304
    Ok well regardless of what the C++ standard standpoint is, the fact is that the volatile keyword is still part of C++, and afaik still has the effect it always had in VS2008, and afaik InterlockedIncrement etc still takes a volatile pointer in VS2008. I don't have VS2008 at home so perhaps someone else could confirm this?
    Now even though you're quite possibly right, Microsoft presumably still feel that it is necessary. Otherwise surely they would take the same route as with 'register', and 'inline' and simply ignore the keyword. (Please let me know if they actually have in VS2008).
    As long as it is still there and still does what it is supposed to, it's hard to imagine that we are never intended to use it. Obviously it is still of some use, or even perhaps necessary in some circumstance, even if that usage is to prevents bugs appearing in their own code, or as a potential workaround for possible compiler bugs. Never say never.

    There's been a lot of talk about implementation defined behaviour too, most of which is implying it is a bad thing. That's not entirely true. Sure it can be good to have standard defined behaviour for portability, but in many areas of C++ there just isn't strict standards around detail of that part of the language. Sometimes the implementation defined behaviour is exactly what you want. Also, sometimes porting the code is just never going to happen anyway.
    My homepage
    Advice: Take only as directed - If symptoms persist, please see your debugger

    Linus Torvalds: "But it clearly is the only right way. The fact that everybody else does it some other way only means that they are wrong"

  8. #23
    C++まいる!Cをこわせ! Elysia's Avatar
    Join Date
    Oct 2007
    Posts
    22,591
    VC9 refuses to cache global variables in a register, it seems:

    Code:
    for (int i = 0; i < 0xFFFFFFFF; i++)
    00401821  xor         esi,esi 
    	{
    		MyGlobalVar++;
    00401823  inc         dword ptr [MyGlobalVar (404384h)] 
    		if (i &#37; 1000) cout << MyGlobalVar << endl;
    00401829  mov         eax,10624DD3h 
    0040182E  imul        esi  
    00401830  sar         edx,6 
    00401833  mov         eax,edx 
    00401835  shr         eax,1Fh 
    00401838  add         eax,edx 
    0040183A  imul        eax,eax,3E8h 
    00401840  mov         ecx,esi 
    00401842  sub         ecx,eax 
    00401844  je          Help+47h (401867h) 
    00401846  mov         edx,dword ptr [__imp_std::endl (402048h)] 
    0040184C  mov         eax,dword ptr [MyGlobalVar (404384h)] 
    00401851  mov         ecx,dword ptr [__imp_std::cout (40204Ch)] 
    00401857  push        edx  
    00401858  push        eax  
    00401859  call        dword ptr [__imp_std::basic_ostream<char,std::char_traits<char> >::operator<< (402040h)] 
    0040185F  mov         ecx,eax 
    00401861  call        dword ptr [__imp_std::basic_ostream<char,std::char_traits<char> >::operator<< (402044h)] 
    00401867  inc         esi  
    00401868  cmp         esi,0FFFFFFFFh 
    0040186B  jb          Help+3 (401823h) 
    0040186D  pop         esi  
    	}
    If, on the other hand, it's a local var, it's cached in a register:

    Code:
    	for (int i = 0; i < 0xFFFFFFFF; i++)
    00401824  xor         esi,esi 
    	{
    		MyLocalVar++;
    		if (i % 1000) cout << MyLocalVar << endl;
    00401826  mov         eax,10624DD3h 
    0040182B  imul        esi  
    0040182D  sar         edx,6 
    00401830  mov         eax,edx 
    00401832  shr         eax,1Fh 
    00401835  add         eax,edx 
    00401837  imul        eax,eax,3E8h 
    0040183D  mov         ecx,esi 
    0040183F  inc         edi  
    00401840  sub         ecx,eax 
    00401842  je          Help+40h (401860h) 
    00401844  mov         edx,dword ptr [__imp_std::endl (402048h)] 
    0040184A  mov         ecx,dword ptr [__imp_std::cout (40204Ch)] 
    00401850  push        edx  
    00401851  push        edi  
    00401852  call        dword ptr [__imp_std::basic_ostream<char,std::char_traits<char> >::operator<< (402040h)] 
    00401858  mov         ecx,eax 
    0040185A  call        dword ptr [__imp_std::basic_ostream<char,std::char_traits<char> >::operator<< (402044h)] 
    00401860  inc         esi  
    00401861  cmp         esi,0FFFFFFFFh 
    00401864  jb          Help+6 (401826h) 
    00401866  pop         edi  
    00401867  pop         esi  
    	}
    Following doesn't work, because even though the variable is volatile, it actually writes the value to a different address than the MyLocalVar, as its value doesn't change.
    I'm not an expert at assembly, so if anyone wants to take a look at what's really happening, feel free. I'll post the assembly code too.

    Code:
    bool bExit = false;
    
    void Thread1(const int& rMyLocalVar)
    {
    	while (!bExit)
    	{
    		cout << rMyLocalVar << endl;
    		Sleep(100);
    	}
    }
    
    void Help()
    {
    	volatile UINT MyLocalVar = 0;
    	CWinThread* pThread;
    	Stuff::NewThreadTF(&pThread, MyLocalVar, &Thread1);
    	for (int i = 0; i < 0xFFFFFFFF; i++)
    	{
    		MyLocalVar++;
    		//if (i % 1000) cout << MyLocalVar << endl;
    	}
    	bExit = true;
    	WaitForSingleObject(pThread->m_hThread, INFINITE);
    }
    Code:
    00401950  sub         esp,8 
    	volatile UINT MyLocalVar = 0;
    00401953  mov         dword ptr [esp],0 
    	CWinThread* pThread;
    	Stuff::NewThreadTF(&pThread, MyLocalVar, &Thread1);
    0040195A  mov         eax,dword ptr [esp] 
    0040195D  push        edi  
    0040195E  push        eax  
    0040195F  lea         edi,[esp+0Ch] 
    00401963  call        Stuff::NewThreadTF<unsigned int,void (__cdecl*)(int const &)> (401000h) 
    00401968  add         esp,4 
    0040196B  or          eax,0FFFFFFFFh 
    0040196E  mov         ecx,1 
    00401973  pop         edi  
    	for (int i = 0; i < 0xFFFFFFFF; i++)
    	{
    		MyLocalVar++;
    00401974  add         dword ptr [esp],ecx 
    00401977  sub         eax,ecx 
    00401979  jne         Help+24h (401974h) 
    		//if (i % 1000) cout << MyLocalVar << endl;
    	}
    	//Stuff::NewThreadTF(&pThread, NULL, &Thread2);
    	//Sleep(3000);
    	bExit = true;
    0040197B  mov         byte ptr [bExit (404384h)],cl 
    	WaitForSingleObject(pThread->m_hThread, INFINITE);
    00401981  mov         ecx,dword ptr [esp+4] 
    00401985  mov         edx,dword ptr [ecx+2Ch] 
    00401988  push        0FFFFFFFFh 
    0040198A  push        edx  
    0040198B  call        dword ptr [__imp__WaitForSingleObject@8 (40202Ch)]
    Last edited by Elysia; 12-30-2007 at 04:36 AM.
    Quote Originally Posted by Adak View Post
    io.h certainly IS included in some modern compilers. It is no longer part of the standard for C, but it is nevertheless, included in the very latest Pelles C versions.
    Quote Originally Posted by Salem View Post
    You mean it's included as a crutch to help ancient programmers limp along without them having to relearn too much.

    Outside of your DOS world, your header file is meaningless.

  9. #24
    Cat without Hat CornedBee's Avatar
    Join Date
    Apr 2003
    Posts
    8,893
    Quote Originally Posted by iMalc View Post
    and afaik InterlockedIncrement etc still takes a volatile pointer in VS2008. I don't have VS2008 at home so perhaps someone else could confirm this?
    InterlockedIncrement is part of the Win32 SDK and independent of VS versions.

    Now even though you're quite possibly right, Microsoft presumably still feel that it is necessary.
    Yes, but possibly simply for source compatibility. You can't just drop the volatile keyword, just as you can't just drop the const keyword.

    Otherwise surely they would take the same route as with 'register', and 'inline' and simply ignore the keyword.
    Ah, no, they can't do that. For single-threaded applications, volatile still has a certain effect required by the standard. Register and inline are just suggestions, volatile is not.

    As long as it is still there and still does what it is supposed to, it's hard to imagine that we are never intended to use it.
    The thing is, there's no agreement on what it is supposed to do in multi-threaded code.

    Quote Originally Posted by Elysia View Post
    Following doesn't work, because even though the variable is volatile, it actually writes the value to a different address than the MyLocalVar, as its value doesn't change.
    Code:
    bool bExit = false;
    
    void Thread1(const int& rMyLocalVar)
    {
    	while (!bExit)
    	{
    		cout << rMyLocalVar << endl;
    		Sleep(100);
    	}
    }
    
    void Help()
    {
    	volatile UINT MyLocalVar = 0;
    	CWinThread* pThread;
    	Stuff::NewThreadTF(&pThread, MyLocalVar, &Thread1);
    	for (int i = 0; i < 0xFFFFFFFF; i++)
    	{
    		MyLocalVar++;
    		//if (i % 1000) cout << MyLocalVar << endl;
    	}
    	bExit = true;
    	WaitForSingleObject(pThread->m_hThread, INFINITE);
    }
    This shouldn't even compile! volatile is sticky, like const. You shouldn't be able to pass the volatlie MyLocalVar to the non-volatile parameter of Thread1. Either VS doesn't enforce this properly, or you have an evil cast going on.
    All the buzzt!
    CornedBee

    "There is not now, nor has there ever been, nor will there ever be, any programming language in which it is the least bit difficult to write bad code."
    - Flon's Law

  10. #25
    C++まいる!Cをこわせ! Elysia's Avatar
    Join Date
    Oct 2007
    Posts
    22,591
    You may be right. I think the compiler is a little poor in this regard. It didn't even catch my mistake of using UINT and passing it as int.
    Anyway, I updated the code, but the problem remains:

    Code:
    bool bExit = false;
    
    void Thread1(volatile const UINT& rMyLocalVar)
    {
    	while (!bExit)
    	{
    		cout << rMyLocalVar << endl;
    		Sleep(100);
    	}
    }
    
    void Help()
    {
    	volatile UINT MyLocalVar = 0;
    	CWinThread* pThread;
    	Stuff::NewThreadTF(&pThread, MyLocalVar, &Thread1);
    	for (int i = 0; i < 0xFFFFFFFF; i++)
    	{
    		MyLocalVar++;
    		//if (i &#37; 1000) cout << MyLocalVar << endl;
    	}
    	bExit = true;
    	WaitForSingleObject(pThread->m_hThread, INFINITE);
    }
    Code:
    00401940  sub         esp,8 
    	volatile UINT MyLocalVar = 0;
    00401943  mov         dword ptr [esp],0 
    	CWinThread* pThread;
    	Stuff::NewThreadTF(&pThread, MyLocalVar, &Thread1);
    0040194A  mov         eax,dword ptr [esp] 
    0040194D  push        edi  
    0040194E  push        eax  
    0040194F  lea         edi,[esp+0Ch] 
    00401953  call        Stuff::NewThreadTF<unsigned int,void (__cdecl*)(unsigned int const volatile &)> (401000h) 
    00401958  add         esp,4 
    0040195B  or          eax,0FFFFFFFFh 
    0040195E  mov         ecx,1 
    00401963  pop         edi  
    	for (int i = 0; i < 0xFFFFFFFF; i++)
    	{
    		MyLocalVar++;
    00401964  add         dword ptr [esp],ecx 
    00401967  sub         eax,ecx 
    00401969  jne         Help+24h (401964h) 
    	}
    	bExit = true;
    0040196B  mov         byte ptr [bExit (404384h)],cl 
    	WaitForSingleObject(pThread->m_hThread, INFINITE);
    00401971  mov         ecx,dword ptr [esp+4] 
    00401975  mov         edx,dword ptr [ecx+2Ch] 
    00401978  push        0FFFFFFFFh 
    0040197A  push        edx  
    0040197B  call        dword ptr [__imp__WaitForSingleObject@8 (40202Ch)]
    (Never used volatile before, so there's my problem I didn't catch a mistake like that )
    Quote Originally Posted by Adak View Post
    io.h certainly IS included in some modern compilers. It is no longer part of the standard for C, but it is nevertheless, included in the very latest Pelles C versions.
    Quote Originally Posted by Salem View Post
    You mean it's included as a crutch to help ancient programmers limp along without them having to relearn too much.

    Outside of your DOS world, your header file is meaningless.

  11. #26
    Registered User Codeplug's Avatar
    Join Date
    Mar 2003
    Posts
    4,669
    Here's one way of writing the same code, but which uses correct synchronization for all Windows compilers and 32bit architectures. It uses just the Win API so anyone with a windows compiler can play with it.

    I pulled InterlockedRead() from my own source. The nice thing about wrapping templates around the Interlocked functions is that you can use template-techniques to automatically choose the 64bit Interlocked API's at compile time (not shown here). Then the code would be correct for all supported Windows architectures.
    Code:
    #include <windows.h>
    
    #include <iostream>
    using namespace std;
    
    /*------------------------------------------------------------------------------
    InterlockedRead - Read a "stable" value in an Interlocked fashion. Returns the
                      the true value of the pointer "at the time" this function
                      returns. This is really only useful for "read with full memory
                      barrier" semantics.
    ------------------------------------------------------------------------------*/
    template<typename T>
    T InterlockedRead(T *psrc)
    {
        T val;
        for (;;)
        {
            val = *psrc;
            if ((T)InterlockedCompareExchange((LONG*)psrc, (LONG)val, (LONG)val)
                    == val)
                break;
        }//if
    
        return val;
    }//InterlockedRead
    
    /*----------------------------------------------------------------------------*/
    
    struct MyThreadParams
    {
        LONG ThreadExit;
        LONG ThreadVal;
    };//MyThreadParams
    
    DWORD WINAPI MyThread(LPVOID param)
    {
        MyThreadParams *pp = reinterpret_cast<MyThreadParams*>(param);
        LONG last_val = -1;
    
        while (!InterlockedRead(&pp->ThreadExit))
        {
            int cur_val = InterlockedRead(&pp->ThreadVal);
            if (cur_val != last_val)
            {
                cout << cur_val << endl;
                last_val = cur_val;
            }//if
    
            Sleep(0);
        }//while
            
        return 0;
    }//MyThread
    
    /*----------------------------------------------------------------------------*/
    
    int main()
    {
        MyThreadParams tparam = {0};
        HANDLE h = CreateThread(0, 0, &MyThread, &tparam, 0, 0);
        if (!h)
        {
            cerr << "CreateThread() failed, ec = " << GetLastError() << endl;
            return 1;
        }//if
        
        for (int n = 0; n < 10; ++n)
        {
            InterlockedIncrement(&tparam.ThreadVal);
            Sleep(750);
        }//for
    
        InterlockedIncrement(&tparam.ThreadExit);
        WaitForSingleObject(h, INFINITE);
    
        cout << "Thread is done" << endl;
    
        return 0;
    }//main
    Keep in mind that volatile is treated just like const. So using the debate angle of "MS put volatile on the Interlocked API's for some reason - so we should use volatile variables" doesn't really make sense. That would be the same as saying "strlen() takes a const pointer, so all variables passed to strlen() should already be const". In other words, non-volatile objects are automatically promoted to volatile objects without a cast - same as with const.

    So the question is - why did MS slap volatile qualifier on the Interlocked pointer parameters? The conclusion reached in many news group threads (and by MS-MVP's) is that is was an unnecessary and poorly conceived change.

    More threds....

    http://groups.google.com/group/comp....4a82dda916b18/
    This is an interesting debate to read. Around post 18 they start to get into what benefits, if any, was MS thinking of when they added volatile to Interlocked API's. Conclusion: there are no benefits, and it promotes and confuses folks into poor MT programming.

    http://groups.google.com/group/micro...f861788a8ecf8f
    I like this thread for 2 reasons, it's a fairly recent thread, and it's another MS-MVP agreeing with me
    Quote Originally Posted by Doug Harrison - Visual C++ MVP
    ...if you're using synchronization, you don't need
    volatile, because synchronization provides the desired memory consistency
    between the threads; indeed, the only thing using volatile on top of
    synchronization does is kill performance.
    gg

  12. #27
    Algorithm Dissector iMalc's Avatar
    Join Date
    Dec 2005
    Location
    New Zealand
    Posts
    6,304
    Quote Originally Posted by CornedBee View Post
    Yes, but possibly simply for source compatibility. You can't just drop the volatile keyword, just as you can't just drop the const keyword.

    Ah, no, they can't do that. For single-threaded applications, volatile still has a certain effect required by the standard. Register and inline are just suggestions, volatile is not.
    Thankyou for your response, and forgive my lack of understanding, but what is volatile actually useful for in single-threaded programming? Oh I suppose it's compiler or platform specific right?

    Sorry, I wasn't actually meaning to suggest dropping the keyword.
    My homepage
    Advice: Take only as directed - If symptoms persist, please see your debugger

    Linus Torvalds: "But it clearly is the only right way. The fact that everybody else does it some other way only means that they are wrong"

  13. #28
    Cat without Hat CornedBee's Avatar
    Join Date
    Apr 2003
    Posts
    8,893
    Using volatile on simple things, like local variables, is extremely useless. You can use volatile for global variables of type sigatomic_t if you want to react to signals. (Standard C and C++ acknowledge the existence of POSIX-like signals, even though they don't provide a way to fire them, or give any information about when one might be fired.)
    You would also probably use volatile for pointers to memory-mapped I/O address ranges. At any time, there might be an interrupt and the device could change the contents of this area. This is stuff that comes from the system-level programming part of C++ and is not really useful in application-level programming.

    Actually, the only time I've used volatile was when I wanted to force the compiler away from some optimizations, e.g. to observe the effect of some code and prevent the compiler from just optimizing it away.
    All the buzzt!
    CornedBee

    "There is not now, nor has there ever been, nor will there ever be, any programming language in which it is the least bit difficult to write bad code."
    - Flon's Law

  14. #29
    Registered User Codeplug's Avatar
    Join Date
    Mar 2003
    Posts
    4,669
    In a previous thread I posted, Alexander Terekhov summed up legal usage of volatile - down in post 42.
    http://groups.google.com/group/comp....1a34f888b85281

    The last time I used volatile (correctly) was in an embedded system using a MIPS CPU. This CPU used memory mapped registers for all the on-CPU functionality. Even the interrupt mask register could be accessed by via a *special* memory address, even though it could be referenced explicitly in assembly. Basically the way it worked is the CPU would "intercept" any access to the address bus if the address was a *special* address - it would then re-route to access to the mapped register. In C, the register could then be directly read and written to using a volatile pointer. The reason the pointer is volatile is so that the compiler can't re-order accesses and so that each access in C code translates to an access to the address bus.

    gg

  15. #30
    Registered User
    Join Date
    May 2006
    Posts
    1,579
    I agree all of your points, CornedBee!


    You know, the conflicting points about this thread is not about the below points, but how should we use volatile in multi-threading programming. :-)

    It is appreciated if you could give us some final insights. :-)

    Quote Originally Posted by CornedBee View Post
    Using volatile on simple things, like local variables, is extremely useless. You can use volatile for global variables of type sigatomic_t if you want to react to signals. (Standard C and C++ acknowledge the existence of POSIX-like signals, even though they don't provide a way to fire them, or give any information about when one might be fired.)
    You would also probably use volatile for pointers to memory-mapped I/O address ranges. At any time, there might be an interrupt and the device could change the contents of this area. This is stuff that comes from the system-level programming part of C++ and is not really useful in application-level programming.

    Actually, the only time I've used volatile was when I wanted to force the compiler away from some optimizations, e.g. to observe the effect of some code and prevent the compiler from just optimizing it away.

    have a good weekend,
    George

Page 2 of 3 FirstFirst 123 LastLast
Popular pages Recent additions subscribe to a feed

Similar Threads

  1. MSDN template sample
    By George2 in forum C++ Programming
    Replies: 12
    Last Post: 03-11-2008, 03:47 AM
  2. MSDN const_cast sample
    By George2 in forum C++ Programming
    Replies: 7
    Last Post: 12-17-2007, 07:32 AM
  3. MSDN OLE DB Sample Provider entry point
    By George2 in forum C++ Programming
    Replies: 0
    Last Post: 07-21-2007, 07:30 AM
  4. Replies: 16
    Last Post: 10-29-2006, 04:04 AM
  5. MSDN Searching Tips
    By jverkoey in forum A Brief History of Cprogramming.com
    Replies: 4
    Last Post: 10-19-2004, 04:51 AM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21