ill try my best to explain it simply while also being in depth.
tldr: use a for loop, dont worry about stack overflows.
your function in assembly & better solution
I recommend you look a godbolt or use some disassembly software to disassemble your c code. ive taken the liberty of doing it for you and commenting the disassembly so you can see what everything does.
Code:
fn_recursion:
; prologue
push ebp ; save previous stack frame ( ebp )
mov ebp, esp ; set current stack frame base ( ebp ) to top of stack.
; if num ( arg1 ) is == 1, return
cmp [ebp+num], 1
jz lbl_goto_end
; if num ( arg1 ) >= 1 do processing ........
cmp [ebp+num], 1
jge lbl_do_processing
; return ( gets executed if num ( arg1 ) == 1 or num ( arg1 ) < 1
lbl_goto_end:
jmp lbl_epilogue
lbl_do_processing:
mov eax, [ebp+num] ; mov num ( arg1 ) to eax
cdq ; prepare signed int in eax for division
mov ecx, 10 ; move the divisor into ecx
idiv ecx ; divide the edx:eax pair by ecx
; do recursion
push eax ; push num ( arg1 )
call fn_recursion ; push return address and transfers control to fn_recursion
add esp, 4 ; restore stack allocation ( push eax )
; epilogue
lbl_epilogue:
pop ebp ; restore stack frame base to previous frame
retn
as you can see lots of stack allocations happen and the generated assembly is rather unideal as calling over and over is much slower than a simple for loop. it requires much more code and many more stack allocations than a for loop.
its much more ideal to do a simple for loop:
Code:
void recursion(int num) {
for (; num >= 1; num /= 10) {
}
}
which looks like:
Code:
fn_recursion:
; prologue
push ebp ; save previous stack frame ( ebp )
mov ebp, esp ; set current stack frame base ( ebp ) to top of stack.
; start loop
jmp lbl_do_loop
; loop code
lbl_division:
mov eax, [ebp+num] ; mov num ( arg1 ) to eax
cdq ; prepare signed int in eax for division
mov ecx, 10 ; move the divisor into ecx
idiv ecx ; divide the edx:eax pair by ecx
mov [ebp+num], eax ; set num ( arg1 ) to divided num ( arg1 )
; handle testing condition and executing loop code
lbl_do_loop:
cmp [ebp+num], 1 ; condition
jl lbl_epilogue ; if condition is met, return
jmp lbl_division ; if condition to return wasnt met, execute loop code
; end / return
lbl_epilogue:
pop ebp ; restore stack frame base to previous frame
retn
checking for overflow & compiler code gen
checking for a stack overflow each time you call the function is also unideal as it slows down code and theoretically a stack overflow could happen anywhere in your code and is unlikely to happen in the provided code due to the small number of stack allocations. its also impossible to completely detect a stack overflow without writing a vm or making your own processor and even if one occured handling it would be pointless.
if for some reason you still wanna check manually for overflows heres how you could do that.
store top of stack in unused register, then check if the current top of stack is > than it on each recursion.
this is really annoying to implement in c however, and you be better of just using the afformentioned for loop.
getting the stack limit
windows:
on windows, you can check the stack limit via the NtThreadInformationBlock (NT_TIB)
Code:
mov eax, fs: 0x18 ; get thread information block from fs data register
; mov eax, [eax+0x4] base of stack memory
; mov eax, [eax+0x8] limit of stack memory
linux:
on linux you can use getrlimit ( ive never used this before and it requires allocation of stack memory to call ).
compiler generated & c runtime
generally stack overflows will be checked for automatically via compile time checks or via compiler generated code such as security cookies, chkstk, amongst other well documented means. for such crt functions you can write your own implementations to handle overflows although its not really recommended.
this code is usually generated when a large amount of data is allocated to the stack ( ex. __chkstk )
chkstk:
__chkstk Routine - Win32 apps | Microsoft Learnsecurity cookies:
__security_init_cookie | Microsoft Learn