Putting code on a diet

[Nick]

Hi all,

I currently have an executable of several megabytes, and I'd like to make it smaller. So the first thing I need to know is where the fat is coming from. I suspect that some libraries I'm linking in have quite a bit of unused functionality, and by breaking unnecessary dependencies I might get a smaller executable. Or maybe there's a huge static array somewhere that I'm not aware of.

So does anyone know of any tools that can help with this?

I'm using Visual C++ 2005 and by letting the linker genere a .map file I get to see the layout of functions in the binary, but it's not convenient for analysis. I do however see that the .text section takes the bulk of the size, so it's unlike to have unnecessary large static data. It would help a lot of the .map data could be sorted by function size and name.

Any thoughts?

Nicolas

[Sol_HSA]

I made a tool once that parses the map file and creates estimates on how large each bit is based on the start addresses. Not exact, but gave some kind of idea. (Unfortunately said tool was proprietary, but wasn't exactly rocket science, should someone reimplement it)

[.oisyn]

If you enable function-level linking, VC++ can remove dead code. Also, try UPX (http://upx.sourceforge.net/) ;)

[Nick]

I made a tool once that parses the map file and creates estimates on how large each bit is based on the start addresses. Not exact, but gave some kind of idea. (Unfortunately said tool was proprietary, but wasn't exactly rocket science, should someone reimplement it)
Yeah, I ended up just writing it myself as well. It's hardly 50 lines of code, I was just lazy. :whistle:

It allowed me to locate the really fat functions and with a bit of tweaking I reduced executable size by 5%. Not a bad start...

[Nick]

If you enable function-level linking, VC++ can remove dead code.
Tried that, but it has no significant effect. I suspect that the linker optimizations already do something just as effective.
Also, try UPX (http://upx.sourceforge.net/) ;)
Heh, UPX is pretty cool, but not what I'm after. I'm already using a compressed installation package. It's the unpacked size in memory I'm most concerned about (think embedded systems) so I need to get to the root of the problem.

Anyway, thanks for the sugggestions, it can probably come in handy for other projects.

[alphadog]

- Are you looking at the size of the Release build vs. the Debug build? Probably yes, but it never hurts to ask... :)
- What libraries are you using? Are you statically or dynamically linking those libraries?
- What are your linker settings? (ex: /FILEALIGN:512 instead of the default 4096)
- What compiler settings are you using?
- Consider use of libctiny.c (http://msdn.microsoft.com/en-us/magazine/cc301696.aspx)
- Consider using StripReloc (http://www.innosetup.com/striprlc.php)

[alphadog]

For example, for linker/compiler setting, try adding in the below.

#ifdef NDEBUG

#pragma optimize("gsy",on)
#pragma comment(linker,"/RELEASE")
#pragma comment(linker,"/merge:.rdata=.data") // merging the .rdata section can result in LARGER exe
#pragma comment(linker,"/merge:.text=.data")
#pragma comment(linker,"/merge:.reloc=.data")
#if _MSC_VER >= 1000
#pragma comment(linker,"/FILEALIGN:0x200")
#endif

#endif // NDEBUG

[Nick]

- Are you looking at the size of the Release build vs. the Debug build? Probably yes, but it never hurts to ask... :)
Release. :happy:
- What libraries are you using? Are you statically or dynamically linking those libraries?
Just a bunch of standard libraries, and some custom ones. I'm statically linking them, as I definitely want to avoid situations where the DLLs are not available on the client system or outdated. Anyhow, my analysis of the .map file shows no prominent presence of standard library functions.
- What are your linker settings? (ex: /FILEALIGN:512 instead of the default 4096)
That corresponds to disabling 'Optimize for Windows 98', right? Yeah I've done that and enabled every other linker optimization.
- What compiler settings are you using?
Optimize for size and disabling inline doesn't have a significant influence, if that's what you mean.

It's not like my executable is hundreds of megabytes. I think the compiler and linker are already doing an excellent job at keeping the size minimal. What I'm really trying to do is locate code that is unnecessarily large. For example the 5% reduction I achieved so far was by removing unused cases out of a pretty large switch statement...

Any tools or techniques to locate more of such situations would be greatly appreciated! :worthy:

[alphadog]

Any tools or techniques to locate more of such situations would be greatly appreciated!

Well, the tool you are looking for is called a "profiler". A profiler would give you not only memory usage, but also point out where you spend most of your execution time, and other interesting tidbits.

As to which exact tool to use, I'm no expert. Depends on whether you want to pay or not. The ones you have to buy tend to be better/more feature-rich. Visual Studio has one built into Team Edition, if that is what you have.

Off the top of my head: Intel VTune (not free?), AMD CodeAnalyst (free), Shiny (free, sourceforge), IBM Quantify (definitely not free), AQTime (not free, but well-priced)... I'm sure there is a lot more.

[Nick]

Well, the tool you are looking for is called a "profiler". A profiler would give you not only memory usage, but also point out where you spend most of your execution time, and other interesting tidbits.
I use CodeAnalyst from time to time. But it doesn't give me the information I'm looking for. I'm not after performance bottlenecks, but code that is unnecessarily large. For example that switch statement had good performance but it made the function's binary code fairly large.
Off the top of my head: Intel VTune (not free?), AMD CodeAnalyst (free), Shiny (free, sourceforge), IBM Quantify (definitely not free), AQTime (not free, but well-priced)... I'm sure there is a lot more.
Thanks, there's a few in there I didn't know of. I'll check 'em out.

[alphadog]

Well, my advice is a little vague because I'm currently dated on profilers. Haven't had to use one in a couple of years.

Generally-speaking, profilers should generally give you information on performance and RAM footprint. Admittedly, they usually emphasize performance, and footprint information is sparse or hard to get. I'm surprised AMD's offering does not, but maybe it's a case of TANSTAAFL? It's such a basic function of a profiler...

A buddy of mine suggested GlowCode.

Another guy here suggested alss looking at MS' xperf. He gave me these links:
http://www.microsoft.com/whdc/system/sysperf/perftools.mspx
http://blogs.msdn.com/pigscanfly/archive/2008/02/09/xperf-a-new-tool-in-the-windows-sdk.aspx

I don't think that last one is what you want, but it's a good profiling tool for any developer.

[.oisyn]

Generally-speaking, profilers should generally give you information on performance and RAM footprint.
Again generally-speaking, in my experience most profilers only concentrate on RAM footprint in terms of runtime memory behaviour, not how much space specific functions take up. However, I've never used profilers for embedded systems and I can imagine it would be a useful feature for such a profiler.

Another guy here suggested alss looking at MS' xperf
Vista+ only :(

[roel]

I use hiew and tdump (from Borland) for things like that. With tdump you can at least see the size of the various sections in the image, to find out whether you have a large idata section (static array) array or not.

Oh, and Crinkler beats UPX big time :) in some cases