Inline assembly : a case study

Our first article "introduction of inline assembly" will coverthe case study of inline assembly and benefit of using inlineassembly in C/C++ program. Of course we will describe the basicsyntax of inline assembly and tell readers how to read and writeinline assembly.

1. Casestudy: thread lock in multi-thread program

With the maturity and popularization of multicore processor and tomaximize the throughput of processors, the latest applications arenormally implemented with multithread to boost the performance.pthread(POSIX thread) is a famous thread library on Linux and is usedby many multithread applications written in c/c++.

As we all know, when writing applications in shared memory model inmultiprocessor environment, we should deal with date synchronization.Pthread library supplies programmers with several kinds of mechanismsto implement data synchronization and one of them is mutual-exclusionlock which is called mutex lock. Check the simple example below. wehave two threads and they will access to a shared variable calledshared_var. one thread will increment it by one and another threadwill decrement it by one. so before accessing the variable functionhas to acquire the lock to ensure on other thread can modify theshared variable.

[cpp] view plaincopy

1. //codesnippet 1   
2.   
3. #include<unistd.h>   
4.   
5. #include<stdio.h>   
6.   
7. #include<pthread.h>   
8.   
9. pthread_mutex_tlock = PTHREAD_MUTEX_INITIALIZER;  
10.   
11. intshared_var;  
12.   
13. void* increment(void * args)  
14.   
15. {  
16.   
17. while(1){  
18.   
19. pthread_mutex_lock(&lock);  
20.   
21. shared_var+=1;  
22.   
23. printf("incre:%d\n",shared_var);  
24.   
25. pthread_mutex_unlock(&lock);  
26.   
27. sleep(0);  
28.   
29. }  
30.   
31. }  
32.   
33.   
34. void* decrement(void * args)  
35.   
36. {  
37.   
38. while(1){  
39.   
40. pthread_mutex_lock(&lock);  
41.   
42. shared_var-=1;  
43.   
44. printf("decre:%d\n",shared_var);  
45.   
46. pthread_mutex_unlock(&lock);  
47.   
48. sleep(0);  
49.   
50. }  
51.   
52. }  
53.   
54.   
55. intmain()  
56.   
57. {  
58.   
59. pthread_t t1,t2;  
60.   
61. pthread_create(&t1,NULL,increment,NULL);  
62.   
63. pthread_create(&t2,NULL,decrement,NULL);  
64.   
65. pthread_join(t1,NULL);  
66.   
67. pthread_join(t2,NULL);  
68.   
69. return 0;  
70.   
71. }  

From the c code above we know that we can use pthread_mutex_lock toacquire a lock and then access shared_variable, which is implementedin c code. But what does pthread_mutex_lock really do in thelow-level.

2.Lock implementation in the low-level hardware

In this section we will have a deep insight of mutual exclusion lockfrom a low-level perspective. Intuitively, it is very easy toimplement a lock. A lock is a shared variable and if the variableequals to 1 it means the lock is locked, otherwise it means that thelock is not locked then we can set the lock to 1 to lock it.

So how Pthread implements a lock. If we hack into the source code ofpthread library then we can get a clear view of the implementation. Alock is implemented with compare_and_swap method, which quiteconforms to our intuition. The code below is extracted from thesource code of pthread library on powerpc architecture.

[cpp] view plaincopy

1. //codesnippet 2   
2.   
3. PT_EIint  
4.   
5. __compare_and_swap(long int *p, long int oldval, long int newval)  
6.   
7. {  
8.   
9. int ret;  
10.   
11.   
12. __asm__ __volatile__ (  
13.   
14. "0: lwarx %0,0,%1 ;"  
15.   
16. " xor. %0,%3,%0;"  
17.   
18. " bne 1f;"  
19.   
20. " stwcx. %2,0,%1;"  
21.   
22. " bne- 0b;"  
23.   
24. "1: "  
25.   
26. : "=&r"(ret)  
27.   
28. : "r"(p), "r"(newval),"r"(oldval)  
29.   
30. : "cr0", "memory");  
31.   
32. /* This version of__compare_and_swap is to be used when acquiring 
33.  
34. a lock, so we don't need toworry about whether other memory 
35.  
36. operations have completed, butwe do need to be sure that any loads 
37.  
38. after this point really occurafter we have acquired the lock. */  
39.   
40. __asm__ __volatile__ ("isync": : : "memory");  
41.   
42. return ret == 0;  
43.   
44. }  

At the first glance we can conclude that it is implemented with assemblybut check it again, it is not just as normal as the assembly we usebefore. Actually it is inline assembly. Inline assembly is a verypowerful feature of modern compiler. It provides C/C++ languageprogrammers abilities to manipulate hardware directly and veryhandy,speedy and powerful in some performance specific and hardwareoperation related applications. So we need learn how to write inlineassembly correctly and efficiently to make our code much powerful.

In the example we are talking about, we want to implement a lock and wewill use compare_and_swap method. According to our convention, 0means unlocked and 1 means locked then we can call__compare_and_swap(lock,0,1) which means

[cpp] view plaincopy

1. If lock == 0 then  
2.   set lock =1 to lock it ;  
3.   return true  
4. elseIf lock == 1 then  
5.   it is locked ;  
6.   return false;  

Now we have already known the internal logic in implementing lock. Sincewe can write a very straightforward pseudo code to implement lock whyshould we bother so much to write complicated inline assembly code inC/C++ program. We all know that when one processor access a sharedvariable in the shared memory another processor might also isaccessing it too, which will cause corruption. So we need lockhardware bus to support an atomic operation. C language is notcapable of realizing this functionality while assembly can because itcan manipulate hardware by instructions. However it is reallycomplicated to write assembly in the system mainly written in C/C++and that is why inline assembly was born. We can easily use assemblyin C/C++ code and do anything assembly can do.it is really cool anduseful.

3.Use of inline assembly

In the last section we have already known what inline assembly can doand why we should use inline assembly and in this section we willintroduce how to use inline assembly based on the example above,including its basic syntax, constraints and clobbered list.

Basic syntax and constraints

First of all, let us see the basic structure of inline assembly written inC/C++ program. All the assembly instructions must be enclosed in__asm__() which tells compiler that we are now using inline assembly.Each assembly instruction must be quoted by double quotation marklike "0: lwarx %0,0,%1 ;".

Maybe you have many questions. for example, what does %0,%1 means in theinstruction, and what are the things starting with colon below theinstructions. This is the distinctive feature of inline assembly usedto bridge inline assemble and C expressions and we will have adetailed description below.

Actually the whole structure of inline assembly is: asm(code : output operandlist : input operand list : clobber list); in our example, outputoperand list is : "=&r"(ret) ; input operand list is:"r"(p), "r"(newval), "r"(oldval) andclobbered list is: "cr0", "memory". don't behurry we will describe one by one.

The output and input operand list can have several entries, separated bycommas. each entry has two items. the first one is a string calledconstraint and the second one is a C expression enclosed inparentheses. That is how C expression is passed into instructions.

Let us talk about constraint at first. Take the the simpler one "r"(p)as an example. What does "r" mean? r means register, morespecific, general register. the whole entry means pass C expression pinto a general register which will be allocated by compiler. Ofcourse,there are many other constraints used to demonstrate the typeof operand but here we just mention the basic one "r"appeared in our example.

Now we know C expression will be passed into inline assembly and compilerwill allocate a register to hold it but how can instructions usethese C expressions. the trick here is the %0,%1 in the instruction "0: lwarx %0,0,%1 ;". Actually all the operands appearedin the output and input list are numbered. The first operand in theoutput operand list is numbered by zero and other operands arenumbered according to the order they appear in the list.

Now you can guess what does %0,%1 in the instructions "0: lwarx%0,0,%1 ;" mean. You are right. %0 means the first operand and%1 means the second operands. All the register reference must bepreceded with % to tell compiler that we are using a operand definedin the input and output list, otherwise compiler will treat it as aconstant value.

Now we have learned a lot, right? It is time to check what the inlineassembly code in the example really does. We describe the behavior ofeach instruction in the comments.

[cpp] view plaincopy

1. __asm____volatile__ (  
2.   
3. "0: lwarx %0,0,%1 ;" //load *p into ret   
4.   
5. " xor. %0,%3,%0;" //xor oldval and *p and store the result into ret.   
6.   
7. " bne 1f;" //if ret != 0 then jump to 1:   
8.   
9. " stwcx. %2,0,%1;" //store newval into *p   
10.   
11. " bne- 0b;" //store fails jump to 0:   
12.   
13. "1: "  
14.   
15. : "=&r"(ret)  
16.   
17. : "r"(p), "r"(newval),"r"(oldval)  
18.   
19. : "cr0", "memory");  

It actually did as what we described with pseudo code in the section 2.In the section 2 we have mentioned lock operation must be supportedby hardware to supply atomic operation. Here instructions lwarx andstwcx. supply an atomic operation so that all the memory accessbetween these two instructions will be locked which can ensure onlyone processor can access shared variable *p.

Clobbered list and volatile

All right, we have almost learned much knowledge about inline assembly.Now we will introduce some advanced skill used in the inlineassembly, that is clobbered list and volatile, which is the verycomplicated and tricky.

These two are both concerned with compiler optimization that is why theyare complicated. Volatile is a modifier following the modifier__asm__ to demonstrate all the instructions in the inline assemblycode block cannot be moved so that they must stay where they are put.Sometimes compiler will think he is more clever than programmers soit could move the instructions to implement some optimization. Atmost time it could lead to a better result but in our exampleactually it could lead to a opposed site. We don't want compiler doany change on our instructions so that we use modifier volatile.

So what about clobbered list? Why should we use clobbered list? From aprogrammer's perspective, actually we do not need it. But forcompiler it really need clobbered list. Why? A clobbered list is usedto tell compiler that which register is used implicitly in the inlineassembly instructions. in our example, instructions xor. and stwcx.can implicitly change the CR0 so that we should specify CR0 in theclobbered list. But why programmer should tell compiler thisinformation? it is concerned with compiler optimization. CR0 does notappear in the instructions explicitly so that compiler might thinkthat CR0 is not changed so that if CR0 is used somewhere latercompiler will not load it again and just store it into the variabledirectly which is wrong because it is actually modified by xor. andstwcx. so that is why programmer should tell compiler that CR0 ismodified and compiler should never reply on its value.

OK I know it is really hard for you to read till now and this one is thelast one. come on!!. the "memory" in the clobbered listmeans that the instructions have modified the memory so that compilershould reload all the variable in the memory after the inlineassembly is executed.

Conclusion:

This is our first article about inline assembly.We think inline assemblyis really cool and really powerful. It provides us more capabilitiesto manipulate hardware and boost performance of our applications.

In this session We have described a real case of inline assembly andanalyzed its basic syntax, constraints and clobbered list. This isjust the first session of our inline assembly article series and wewill continue to introduce inline assembly in the following articles.Please enjoy it.

Reference:

1. GCC-Inline-Assembly-HOWTO

2. ARM GCC Inline Assembler Cookbook

3. POSIX thread: http://ftp./gnu/glibc/

4. https://computing./tutorials/pthreads/