The difference between the program headers on 64-bit is for layout. The fields in the elf headers are aligned to their natural alignment, so 8-byte types are likely going to want to be aligned to 64-bit. It was done to move the 32-bit members together so the header wouldn't have to contain unnecessary padding.

LOL I just watched the video, saw the part where he explains the difference and immediately had the same idea. I paused, thinking "I wonder if..." and found your comment right away. :-)


0:47 segment and section differentiation
2:05 ELF structure
3:39 ELF header data structure
9:34 Program header data structure
13:54 Section header data structure

Thanks a lot, I put it in the description!


Incredible how a format developed so much time ago is still capable of being fully functional without any need of a version change. That tells a lot on the foresight of the creators!

WebAssembly is another format that I think is well designed and will last well into the future.

I agree.  The Unix design is quite elegant, some have called it timeless.


13:00 Ah, the infamous `p_align` field that everyone understands wrong :P  This is not about 4-byte vs. 8-byte alignment (and you would know that if you ever tried hand-crafting ELF files or fiddling with existing ones), but it is meant to facilitate paging . Many people say that segments have to be aligned to page boundaries, but this is also wrong! The only true requirement is that their offsets in file and in memory must be congruent modulo the `p_align` value (which is usually some reasonable multiple of the page size). The reason for this is that when segments from the file are memory-mapped (using `mmap`), the granularity of that mapping is the page size (i.e. you can only map whole pages), but the address is usually assigned by the kernel, and it might change on different runs. This isn't a problem, though, as long as data and instructions keep the same offsets from the page boundary. Then they can be easily mapped to different addresses in different address spaces, and yet their offsets from page boundaries stay the same as they were in the segment in the file. If the offset in the file were different than in memory, the system loader would have to shift it so that the offsets matched again, but then the beginning of the segment in memory would have to cross the page boundary, so the loader would have to map an additional page in front of our page, with lots of padding at the beginning. It would impact performance, memory usage, and be much more complicated to implement. Hence the alignment criterion that makes sure that when segments are mapped into memory, the offsets in the file and in memory match each other (i.e. they're the same  modulo `p_align`).
13:16 This is because of machine word alignment. Two `Word`s are 64-bit together, so the next field will be aligned to 64-bit machine word boundary. Otherwise a padding would be needed which would only waste space. But since this padding is precisely the size of `p_flags`, they decided to rather move the `p_flags` field in there than waste this space for paddings.

13:00 Ah, the infamous `p_align` field that *everyone understands wrong* :P  This is _not_ about 4-byte vs. 8-byte alignment (and you would know that if you ever tried hand-crafting ELF files or fiddling with existing ones), but it is meant to facilitate *paging* . Many people say that segments have to be aligned to page boundaries, but *this is also wrong!* The only true requirement is that their _offsets_ in file and in memory must be *congruent modulo the `p_align` value* (which is usually some reasonable multiple of the page size). The reason for this is that when segments from the file are memory-mapped (using `mmap`), the granularity of that mapping is the page size (i.e. you can only map whole pages), but the address is usually assigned by the kernel, and it might change on different runs. This isn't a problem, though, as long as data and instructions keep the same offsets from the page boundary. Then they can be easily mapped to different addresses in different address spaces, and yet their offsets from page boundaries stay the same as they were in the segment in the file. If the offset in the file were different than in memory, the system loader would have to shift it so that the offsets matched again, but then the beginning of the segment in memory would have to cross the page boundary, so the loader would have to map an additional page in front of our page, with lots of padding at the beginning. It would impact performance, memory usage, and be much more complicated to implement. Hence the alignment criterion that makes sure that when segments are mapped into memory, the offsets in the file and in memory match each other (i.e. they're the same  modulo `p_align`).
13:16 This is because of machine word alignment. Two `Word`s are 64-bit together, so the next field will be aligned to 64-bit machine word boundary. Otherwise a padding would be needed which would only waste space. But since this padding is precisely the size of `p_flags`, they decided to rather move the `p_flags` field in there than waste this space for paddings.


Oh, acturally ELF is not exactly the executable format of the PS4,

Rather 'SELF' which is like ELF but with some extra stuff added at the start mostly some boot flags, authentication & permissions info if its unencrypted (only on devkits) etc and then the actural elf header and stuff

Though it's often pretty easy to strip away the SELF stuff (and sometimes decrypt its contents too..) which basically just converts your SELF into a standard ELF binary-  but the console cannot run ELF directly !

Oh also SELF is used on the PSVita and PS3 as well. and the format of SELF is different on all there consoles..

Lol on PSVita SELFs acturally have 2 ELF headers lol not sure about PS4 though.

Only PS2 and PS1 use ELF directly!

Oh, acturally ELF is *not exactly* the executable format of the PS4,

Rather 'SELF' which is like ELF but with some extra stuff added at the start mostly some boot flags, authentication & permissions info if its unencrypted (only on devkits) etc and then the actural elf header and stuff

Though it's often pretty easy to strip away the SELF stuff (and sometimes decrypt its contents too..) which basically just converts your SELF into a standard ELF binary-  but the console cannot run ELF directly !

Oh also SELF is used on the PSVita and PS3 as well. and the format of SELF is different on all there consoles..

Lol on PSVita SELFs acturally have 2 ELF headers lol not sure about PS4 though.

Only PS2 and PS1 use ELF directly!

@Silica 🏳⚧ True true. That's irrelevant to ELF.

@Robert Kiestov i like to see cute anime girls die i find it cute for cute people to die. and this is irrelevant to the fact that ps4 and psvita, etc do not use ELF directly.

@Silica 🏳⚧ Nobody will ever take a tranime poster seriously. If you want anyone to listen to you, drop the childish obsessions.

@Null Well, that and to preserve my PS2's system laser. They don't last forever...

@Xeno Idaltu Just do a Free McBoot search on YouTube they're already out there... An old programmer's saying: "why reinvent the wheel?"

@skilz8098

Interesting.

You should upload a tutorial!

That's how I was able to load Free McBoot onto one of my PS2 memory cards without having to `hard-mod` the system! It is a process though as you need a specific game title that you have to rip from disc using specific software to a computer, then you need different software to modify the copy, and another piece of software to patch the modified copy, a dvd burner with a specific program to burn it properly. It does involve taking the PS2 apart and removing a spring, and blocking a couple of push buttons to stop indicating that the tray is opened. This works for the "slim" model only, there are other methods for the fat model... Then you will need a blank memory card, a USB thumb drive less than 1GB preferably 256 - 521 MB as the USB ports on the PS2 only support USB 1, maybe USB2 with some of the newest versions of the slim models... and they don't support large size drives that you see today such as 8-256 GB. You will then have to play that specific title up to a certain point, save the game at a certain point, and then when you go to proceed to another game state such as a loading screen to the next level, you have to pop the original disc out with the tray open and while the system is still spinning because it believes the tray is closed, you will then have to pop in your burnt version of the game that will have the `elf` file that you added to that game disc within a specific track and sector... Then if done properly, the console will read that `elf` file and execute it. This acts like a `flash bios operating system - file manager`. Now with this elf running, you can now insert your flash drive that has other elf files... It's been a while since I've done this, but you either move them over to the memory card directly, or you "run" the Free McBoot directly from the USB as it will format your memory card, you can also have other applications and utilities on that USB such as media players, NES, SNES, Sega Emulators and Roms, and more. Once you have Free McBoot installed onto the memory card, you can then shut the system down, remove the discs and USB drive... start the system back up, with the modified memory card. Now, when your PS2 loads the main menu that you would see without any discs have changed. You can customize this list of applications with the same elf file that you used from the modified game disc, however, you will have a copy of this on your memory card so you won't have to do the hot disc swap trick. From there I was able to install, I believe it's called OPL2 or something close to that... and I configured one of my hard drives on my PC to have `ISO` images of all my game discs. This is another long process to setup... but I can now run any of my PS2 and PS1 games from computers had drive across my network. This process is involved too...  Even though I am a Windows user when it comes to PCs mostly due to "Gaming" reasons and that most of my systems came with Windows... Now, I don't mind Linux, but I can't stand Apple!   But yes, that is the power of ELF unlike Window's EXE!

@Silica 🏳⚧ Ah, you are right, I forgot the PSP does have an encryption scheme for its executables too.<br>Welp, that only leaves PS2 in the "it runs ELF files directly" gang.

@Silica 🏳⚧ Ah, you are right, I forgot the PSP does have an encryption scheme for its executables too.
Welp, that only leaves PS2 in the "it runs ELF files directly" gang.

@Eder Barrero nope PSP is self too (see ~PSP header) on UMDs or its PBP for digital stuff

As far as I know the PS1 uses EXE's, they are kind of based off the ELF format, but simpler. The PSP does use plain old ELF's, iirc.


Very well presented video, it would be interesting to see more details on the symbol table and how it compares to PE binaries and pdb files on Windows.


Great explanation! I love digging into executable formats... wrote my own disassembler and have PE/COFF/ELF/etc loaders.  My Elf reader code converts the ELF32 structures to ELF64.


Very nice explanation! Definitely helpful and clearly explained! I would love to see/hear more from the reverse engineer's perspective like when you demonstrated the "sstrip" tool and that an ELF can still run without section headers. In other words things that we take for granted in a normal workflow but during reversing they may appear differently.


The order of the ProgramHeader is surely (my guess not confirmed knowledge) different between 32 and 64 so that the data aligns to 32 respective 64bit boundaries. For 32bit you have: Word = 32, Offset = 32, ... so all is fine, but for 64bit that does not work: Word=32, Offset=64. Thus, you want to have: Word+Word=32+32=64, Offset = 64. I guess the 32bit version wasn't changed to not break backwards compatibility


Awesome tutorial! Haven't looked into what happens inside an ELF since the time when Amigas used 68k and PowerPC simultaneously!


Please keep making videos like this, all of your videos are so helpful and the topics are explained so well! You are awesome man, I have not absorbed info like this in a while.


Thank you for the well presented, well written video. Keep up the good work!


Subscribed!
always a good day when you learn something new.

Just curious - if sections are only used during compiling, and removing them doesn't affect the execution of the program, why aren't they stripped out as part of the compile process?

(I'm presuming this only applies to executable ELF files - unsure if an object or library still needs these, but I can kinda see that they would...)


Very nice explanation, clear, and directly to the point! Thank you!


Really interesting video!<br>If anyone is interested in learning about the execution of programs in Linux, I recommend a series of articles by LWN titled 'How programs get run', and its follow up 'How programs get run: ELF binaries'. It explains how the OS loads executables into memory and how it invokes an interpreter on them if needed (eg. the shell for scripts, or the ELF program loader for dynamically-link programs).

Really interesting video!
If anyone is interested in learning about the execution of programs in Linux, I recommend a series of articles by LWN titled 'How programs get run', and its follow up 'How programs get run: ELF binaries'. It explains how the OS loads executables into memory and how it invokes an interpreter on them if needed (eg. the shell for scripts, or the ELF program loader for dynamically-link programs).


Great explanation, thank you. Definitely keeping this in my favorites list


This video was so well made and thought out! Kudos! :)


Very concise and precise overview of the ELF format. One of the best, if not best summary of the elf, program and section headers


Awesome Work. I really enjoy these deeply technical videos, keep up with them.


Thanks for a hugely informative description. I've been working with elf files for years without fully understanding the structure. Great job.


It would be good to see a video on reverse engineering a React Native app for Android, iOS, or both. Especially one that just uses a WebView. Should be very easy but I'm struggling.


nice animations and explanation - must have taking a long time to make 👍


the editing style reminds me of retro games mechanics explained! nice explanations, though maybe a touch too fast-paced in explanations

Glad you liked it! Yeah, it's definitely difficult to find a good pace :)


Amazing . Densely packed , saved a lot of time


i just started writing a bootloader and this is exactly what i need :)

Well, I guess it's a second stage bootloader then.

@Squidwayne Johnswardson Probably stuff like loading kernels, as you can use ELF for that

What is your bootloader doing with ELF? I am genuinely interested.


Thank you! I'm trying to make a compiler and this is very helpful. The System V spec is very detailed and it's not the best thing to start up for beginners.


It's also used in PS2 games and PSP games.  Both the main executable on the PS2 disc and the boot.bin (once decrypted) on PSP umds are standard ELFs.


Nice video i literally just started studying about executable file formats and then i see you uploaded this.<br>Will you do one about the PE format next? I heard its kinda complicated compared to ELF so it would be a big help

Nice video i literally just started studying about executable file formats and then i see you uploaded this.
Will you do one about the PE format next? I heard its kinda complicated compared to ELF so it would be a big help

Stavsen ⸻⸻⸻⸻⸻⸻⸻⸻⸻⸻⸻⸻⸻⸻⸻⸻⸻⸻⸻⸻⸻⸻⸻⸻⸻⸻⸻⸻⸻⸻⸻⸻⸻⸻⸻⸻

@Lawrence D’Oliveiro The EULA can't modify law its invalid then.

So when you click “agree” on the EULA, you are really doing it under false pretences?

You are not allowed to share the binaries, reverse engineering is still completely valid :) it’s even allowed to reverse engineer to find trade secrets etc

@stacksmashing I'm not a lawyer but I think this may not be entirely correct because of copyright law. But I don't know if this applies for private use.

In a ton of countries such EULAs are not lawful. (For example in Germany you have the right to reverse engineer software.)

Most of those programs also have EULAs that prohibit reverse-engineering, decompiling and general snooping about.

@Lawrence D’Oliveiro thats actually exactly why im so interested in it. So many closed source programs are windows only and so knowing the windows executable format would be helpful in reverse engineering. Most ELF programs are open source anyway :p

PE is Windows-only. It’s not used on any other platforms.

Yeah I agree too. A video on PE file format would be great help.

Wow cool nickname (even screws with the site layout a little bit). How did you do it? (Especially the line break)


Please do a video about relocations, is such a important topic about ELF format! That one was pretty good walkthrough.


Hello to ITMO students, who are trying to write an elf parser for Risc-V. And thanks for an excellent video explanation.


I'm developing my own operating system, and this is EXACTLY what i need.

Might yoink elf for my weird graphing calculator bytecode virtual machine thing.


Have you looked at how, say, it compares to COFF? I recall that was popular among some Unix vendors back in the day. IBM (of course) created their own variant, called XCOFF, which Apple adopted for use on the PowerPC Macs.


great work. it really helps to understand elf


Nice video! Thanks also for the further readings!


Underrated channel, it's an awesome video (very useful for the ctf i m doing right now xD), great job ! Can't wait for other videos.


10:58 you said that only executable will have PT_INTERP, but in fact, the shared libraries (e.g. libc.so) also have PT_INTERP because they may link other libs as well. Correct?


Nice Video and Awesome Explanation thank you so much :)


Looking forward for the next videos!! great job :)


A few time you mentioned some differences between 32 and 64 bit. A bit sad that you didn't said how it is on other systems, like 16 bit and 8 bit. My MSP430 is a 16 bit processor and my AVR is a 8 bit processor, i use GCC and ELF-Files for both of them.

@stacksmashing Thank you, i didn't know that. It would probably help to mention that because it is easy to confuse ELF bit width with architecture bit with.

And they are either 32 bit or 64 bit ELF files :) it says nothing about the contents of the sections, but about the addressing used in the file.


One of the few youtube videos where you have to reduce the playback speed. Great Video !!


Great video !!!!    Looking forward to your update of ELF!!!


That was good, now please do as many other file formats as possible (executables or not), including proprietary ones


All these years, I've been thinking ELF was "a Linux thing".


KDE has a new program called ELF Dissector.

Please review it!


What is different between e_version and EI_VERSION?


Wii stuff on the middle and end of it's generation moved from elf to dol, also iirc the gamecube also uses elf sometimes


Long time .
Learned alot .
Next PE please .


How can I encrypted ELF 64-bit /32-bit in my python script.. Full vedio needed


ELF was also used in the ps1 and ps2 afaik.


Why can't I read the segments and sections using their c struct from withing the program?


ELF: headers ok<br>PE: "This program cannot be run in DOS mode."

ELF: headers ok
PE: "This program cannot be run in DOS mode."


an extensible video about the extensible and linkable format


A reverse engineer expert for a hack group called "CODEX" pointed me to this video.


PT_SHLIB is in fact in the spec. It's just undefined as of right now, so you can do whatever you like with it, but "here be dragons".

In that, there's no guarantee that you'll stay conformant if it changes. So yeah.


Fun fact elf is used under brewmp.

Sometimes


In depth ELF: <br>Elf is the german word for the number eleven, which comes after zehn and before zwölf.

In depth ELF:
Elf is the german word for the number eleven, which comes after zehn and before zwölf.


oof, the intro shows the E being "Executable", but in the title, it's "Extensible". Voice says "Extensible". Executable is right though lol

@stacksmashing Oh, seems like it actually was formerly called Extensible, but now it's Executable.

Oof, thanks! It used to be called Extensible, must’ve gotten confused eventually :)


12:56 You can have a segment which is both writable and executable. For example, this can happen to your program stack segment if you use the GNU C extension of declaring one function inside another.


... are you saying that the Wii and the PS4 share the same executable format?


5:23 I suppose one should distinguish between revisions to the fundamental architectural ABI, versus revisions to the ABI for this particular library (public struct layout changes etc). Is this for the former?


Hopefully you can make a video on hacking the game and watch


If you know the existanse of memtest86.elf you know something about /boot


I remember writing an elf parser in a coffee shop when I was homeless.  Good times.


The video title is incorrect. Executable and Linkable Format, not extensible.


2:33 A.k.a “BSS”. That’s an acronym from the early days of Unix.


How can i thia encryption, pls full vedio needed and WITH script installing full vedio.pls pls upload it


I didn't understand the part of et_exec : it is doesn't support position independsnt executables? That is weird because I can compile any program without -no-pie and I can execute it on my system!

The type ET_EXEC only supports position dependent code. So when you compile with -no-pie the ELF file will be of type ET_EXEC. If you compile with PIE it will be ET_DYN.


13:17 Obviously done for alignment reasons.


My mind just blew up at the half time, it was too much information O_O

Especially without explaining what things are. If I don't know what something does or is used for, it is pretty useless to know where in a file it is. It would have been better to make a series and tackle 1 part in each video. This video should have been split into at least 6 videos.


I was reading the Practical Binary Analysis by Dennis Andriesse when this video arrived...


0:08 well what is it, "executable" or "extendible"?

Yes!😄

It used to be extensible and now it’s executable and naturally between talking about it and making the thumbnail I got confused 😅


So, basically, its a messy context-sensitive grammar? Or is ELF actually Turing-complete?

Didn't expect to find another pony here :)  /)<br>As for your question: I don't think ELF files have anything to do with grammars. ELF is just a container for binary data. How those data are interpreted is up to the loaders that load and interpret them. So Turing-completeness doesn't quite apply here as well.

Didn't expect to find another pony here :)  /)
As for your question: I don't think ELF files have anything to do with grammars. ELF is just a container for binary data. How those data are interpreted is up to the loaders that load and interpret them. So Turing-completeness doesn't quite apply here as well.


Now stop there, an ps2 has one .ELF File for every game...


Too extensible, and waaay too much linking.


So the sectors, which hold the code and data, are loaded to memory inside segments. But nevertheless those can be stripped and irrelevant in runtime. It sounds like there's some kind of contradiction.


Me.Exe
Her.Elf
Impossible to compile :'v


The last time I watched your videos you were ghidra ninja


"Most BSD's and the playstation 4" is a little reduntant to say; since PS4 is in the category of "most BSD's" or, a BSD within "most BSD's" - so is PS3. As far as I understand, Playstation 2 is not, entirely at least, but still also utilized ELF as well and by logic while not the only format for executable software, Wii's compatibility with ELF should also stretch back to gamecube and be more usable on that platform.

Also most software as far as I (as a hobbyist, and user rather than developer) understand, are preferred to be in DOL format on these two "very different" (joke: they're not) systems. ELF is used but mainly for debugging or older homebrew and what short research showed the mainly used executable format of Wii and Gamecube is not even a cousin to ELF, but DOL can be derived out of an ELF, making it probably more a debugging option for software rather than main format like with PS2, PS3, PS4, PS5.....


It's not "Extensible", though. Why didn't you unabbreviated it correct?

@stacksmashing so, yeah, it was very long time ago. nobody calls it "Extensible" for at least 20 years already.

Executable and Linkable Format (ELF, formerly named Extensible Linking Format),


3:09 More than you ever wanted to know about shared libraries: https://akkadia.org/drepper/dsohowto.pdf


Hows the Nintendo game & watch coming along?


Not at all an ‘in depth’ ... it is basically a reading of the doc