Solaris X86 64-bit Assembly Programming
This is a simple example on writing, compiling, and debugging Solaris 64-bit x86 assembly language with a C program.
This is also referred to as "AMD64" assembly.
The term "AMD64" is used in an inclusive sense to refer to all X86 64-bit processors, whether AMD Opteron family or Intel 64 processor family. Both run Solaris x86.
I'm keeping this example simple mainly to illustrate how everything comes together—compiler, assembler, linker, and debugger when using assembly language.
The example I'm using here is a C program that calls an assembly language program passing a C string.
The assembly language program takes
the C string and calls printf() with it to print
the string.
AMD64 Register Usage
But first let's review
the use of AMD64 registers.
AMD64 has several 64-bit registers, some special purpose (such as
the stack pointer) and others general purpose. By convention, Solaris follows
the AMD64 ABI in register usage, which is
the same used by Linux, but different from Microsoft Windows in usage (such as which registers are used to pass parameters). This blog will only discuss conventions for Linux and Solaris.
The following chart shows how AMD64 registers are used.
The first six parameters to a function are passed through registers.
If there's more than six parameters, parameter 7 and above are pushed on
the stack before calling
the function.
The stack is also used to save temporary "stack" variables for use by a function.
64-bit Register
Usage
%rip
Instruction Pointer points to
the current instruction
%rsp
Stack Pointer
%rbp
Frame Pointer (saved stack pointer pointing to parameters on stack)
%rdi
Function Parameter 1
%rsi
Function Parameter 2
%rdx
Function Parameter 3
%rcx
Function Parameter 4
%r8
Function Parameter 5
%r9
Function Parameter 6
%rax
Function return value
%r10, %r11
Temporary registers (need not be saved before used)
%rbx, %r12, %r13, %r14, %r15
Temporary registers, but must be saved before use and
restored before returning from
the current function
(usually with
the push and pop instructions).
32-, 16-, and 8-bit registers
To access
the lower 32-, 16-, or 8-bits of a 64-bit register use
the following:
64-bit register
Least significant 32-bits
Least significant 16-bits
Least significant 8-bits
%rax%eax%ax%al
%rbx%ebx%bx%bl
%rcx%ecx%cx%cl
%rdx%edx%dx%dl
%rsi%esi%si%sil
%rdi%edi%di%axl
%rbp%ebp%bp%bp
%rsp%esp%sp%spl
%r9%r9d%r9w%r9b
%r10%r10d%r10w%r10b
%r11%r11d%r11w%r11b
%r12%r12d%r12w%r12b
%r13%r13d%r13w%r13b
%r14%r14d%r14w%r14b
%r15%r15d%r15w%r15b
%r16%r16d%r16w%r16b
There's other registers present, such as
the 64-bit %mm registers, 128-bit %xmm registers, 256-bit %ymm registers, and 512-bit %zmm registers. Except for %mm registers, these registers may not present on older AMD64 processors.
Assembly Source
The following is
the source for a C program, helloas1.c, that calls an assembly function, hello_asm().
$ cat helloas1.c
extern void hello_asm(char *s);
int
main(void)
{
hello_asm("Hello, World!");
}
The assembly function called above, hello_asm(), is defined below.
$ cat helloas2.s
/*
* helloas2.s
* To build:
* cc -m64 -o helloas2-cpp.s -D_ASM -E helloas2.s
* cc -m64 -c -o helloas2.o helloas2-cpp.s
*/
#if defined(lint) || defined(__lint)
/* ARGSUSED */
void
hello_asm(char *s)
{
}
#else /* lint */
#include <sys/asm_linkage.h>
.extern printf
ENTRY_NP(hello_asm)
// Setup printf parameters on stack
mov %rdi, %rsi // P2 (%rsi) is string variable
lea .printf_string, %rdi // P1 (%rdi) is printf format string
call printf
ret
SET_SIZE(hello_asm)
// Read-only data
.text
.align 16
.type .printf_string, @object
.printf_string:
.ascii "
The string is: %s.\n\0"
#endif /* lint || __lint */
In
the assembly source above,
the C skeleton code under "#if defined(lint)" is optionally used for lint to check
the interfaces with your C program--very useful to catch nasty interface bugs.
The "asm_linkage.h" file includes some handy macros useful for assembly, such as ENTRY_NP(), used to define a program entry point, and SET_SIZE(), used to set
the function size in
the symbol table.
The function hello_asm calls C function printf() by passing two parameters, Parameter 1 (P1) is a printf format string, and P2 is a string variable.
The function begins by moving %rdi, which contains Parameter 1 (P1) passed hello_asm, to printf()'s P2, %rsi. Then it sets printf's P1,
the format string, by loading
the address
the address of
the format string in %rdi, P1. Finally it calls printf.
After returning from printf,
the hello_asm function returns itself.
Larger, more complex assembly functions usually do more setup than
the example above. If a function is returning a value, it would set %rax to
the return value. Also, it's typical for a function to save
the %rbp and %rsp registers of
the calling function and to restore these registers before returning. %rsp contains
the stack pointer and %rbp contains
the frame pointer. Here is
the typical function setup and return sequence for a function:
ENTRY_NP(sample_assembly_function)
push %rbp // save frame pointer on stack
mov %rsp, %rbp // save stack pointer in frame pointer
xor %rax, %r4ax // set function return value to 0.
mov %rbp, %rsp // restore stack pointer
pop %rbp // restore frame pointer
ret // return to calling function
SET_SIZE(sample_assembly_function)
Compiling and Running Assembly
Use
the Solaris cc command to compile both C and assembly source, and to pre-process assembly source. You can also use GNU gcc instead of cc to compile, if you prefer.
The "-m64" option tells
the compiler to compile in 64-bit address mode (instead of 32-bit).
$ cc -m64 -o helloas2-cpp.s -D_ASM -E helloas2.s
$ cc -m64 -c -o helloas2.o helloas2-cpp.s
$ cc -m64 -c helloas1.c
$ cc -m64 -o hello-asm helloas1.o helloas2.o
$ file hello-asm helloas1.o helloas2.o
hello-asm: ELF 64-bit LSB executable AMD64 Version 1 [SSE FXSR FPU], dynamically linked, not stripped
helloas1.o: ELF 64-bit LSB relocatable AMD64 Version 1
helloas2.o: ELF 64-bit LSB relocatable AMD64 Version 1
$ hello-asm
The string is: Hello, World!.
Debugging Assembly with MDB
MDB is
the Solaris system debugger. It can also be used to debug user programs, including assembly and C.
The following example runs
the above program, hello-asm, under control of
the debugger.
In
the example below I load
the program, set a breakpoint at
the assembly function hello_asm, display
the registers and
the first parameter, step through
the assembly function, and continue execution.
$ mdb hello-asm # Start
the debugger
> hello_asm:b # Set a breakpoint
> ::run # Run
the program under
the debugger
mdb: stop at hello_asm
mdb: target stopped at:
hello_asm: movq %rdi,%rsi
> $C # display function stack
ffff80ffbffff6e0 hello_asm()
ffff80ffbffff6f0 0x400adc()
> $r # display registers
%rax = 0x0000000000000000 %r8 = 0x0000000000000000
%rbx = 0xffff80ffbf7f8e70 %r9 = 0x0000000000000000
%rcx = 0x0000000000000000 %r10 = 0x0000000000000000
%rdx = 0xffff80ffbffff718 %r11 = 0xffff80ffbf537db8
%rsi = 0xffff80ffbffff708 %r12 = 0x0000000000000000
%rdi = 0x0000000000400cf8 %r13 = 0x0000000000000000
%r14 = 0x0000000000000000
%r15 = 0x0000000000000000
%cs = 0x0053 %fs = 0x0000 %gs = 0x0000
%ds = 0x0000 %es = 0x0000 %ss = 0x004b
%rip = 0x0000000000400c70 hello_asm
%rbp = 0xffff80ffbffff6e0
%rsp = 0xffff80ffbffff6c8
%rflags = 0x00000282
id=0 vip=0 vif=0 ac=0 vm=0 rf=0 nt=0 iopl=0x0
status=<of,df,IF,tf,SF,zf,af,pf,cf>
%gsbase = 0x0000000000000000
%fsbase = 0xffff80ffbf782a40
%trapno = 0x3
%err = 0x0
> ::dis # disassemble
the current instructions
hello_asm: movq %rdi,%rsi
hello_asm+3: leaq 0x400c90,%rdi
hello_asm+0xb: call -0x220 <PLT:printf>
hello_asm+0x10: ret
0x400c81: nop
0x400c85: nop
0x400c88: nop
0x400c8c: nop
0x400c90: pushq %rsp
0x400c91: pushq $0x74732065
0x400c96: jb +0x69 <0x400d01>
> 0x0000000000400cf8/S # %rdi contains Parameter 1
0x400cf8: Hello, World!
> [ # Step and execute 1 instruction
mdb: target stopped at:
hello_asm+3: leaq 0x400c90,%rdi
> [
mdb: target stopped at:
hello_asm+0xb: call -0x220 <PLT:printf>
> [
The string is: Hello, World!.
mdb: target stopped at:
hello_asm+0x10: ret
> [
mdb: target stopped at:
main+0x19: movl $0x0,-0x4(%rbp)
> :c # continue program execution
mdb: target has terminated
> $q # quit
the MDB debugger
$
In
the example above, at
the start of function hello_asm(), I display
the stack contents with "$C", display
the registers contents with "$r", then disassemble
the current function with "::dis".
The first function parameter, which is a C string, is passed by reference with
the string address in %rdi (see
the register usage chart above).
The address is 0x400cf8, so I print
the value of
the string with
the "/S" MDB command: "0x0000000000400cf8/S".
I can also print
the contents at an address in several other formats.
Here's a few popular formats. For more, see
the mdb(1) man page for details.
address/S C string
address/C ASCII character (1 byte)
address/E unsigned decimal (8 bytes)
address/U unsigned decimal (4 bytes)
address/D signed decimal (4 bytes)
address/J hexadecimal (8 bytes)
address/X hexadecimal (4 bytes)
address/B hexadecimal (1 bytes)
address/K pointer in hexadecimal (4 or 8 bytes)
address/I disassembled instruction
Finally, I step through each machine instruction with
the "[" command, which steps over functions. If I wanted to enter a function, I would use
the "]" command. Then I continue program execution with ":c", which continues until
the program terminates.
MDB Basic Cheat Sheet
Here's a brief cheat sheet of some of
the more common MDB commands useful for assembly debugging.
There's an entire set of macros and more powerful commands, especially some for debugging
the Solaris kernel, but that's beyond
the scope of this example.
$C Display function stack with pointers
$c Display function stack
$e Display external function names
$v Display non-zero variables and registers
$r Display registers
::fpregs Display floating point (or "media" registers). Includes %st, %xmm, and %ymm registers.
::status Display program status
::run Run
the program (followed by optional command line parameters)
$q Quit
the debugger
address:b Set a breakpoint
address:d Delete a breakpoint
$b Display breakpoints
:c Continue program execution after a breakpoint
[ Step 1 instruction, but step over function calls
] Step 1 instruction
address::dis Disassemble instructions at an address
::events Display events
Further Information
"Assembly Language Techniques for Oracle Solaris on x86 Platforms" by Paul Lowik (2004).
Good tutorial on Solaris x86 optimization with assembly.
The Solaris Operating System on x86 Platforms
An excellent, detailed tutorial on X86 architecture, with Solaris specifics.
By an ex-Sun employee, Frank Hofmann (2005).
"AMD64 ABI Features", Solaris 64-bit Developer's Guide
contains rules on data types and register usage for Intel 64/AMD64-class processors.
(available at docs.oracle.com)
Solaris X86 Assembly Language Reference Manual
(available at docs.oracle.com)
SPARC Assembly Language Reference Manual
(available at docs.oracle.com)
System V Application Binary Interface (2003) defines
the AMD64 ABI for UNIX-class operating systems, including Solaris, Linux, and BSD.
Google for it—the original website is gone.
cc(1), gcc(1), and mdb(1) man pages.
