GNU libc 2.16 build fails

[Raxor]

I tried to get the new glibc 2.16 working... make didn't fail, but at make check:
tests/dlfcn: Error 2
Didn't find any useful information about it.

I also installed it several times:
../glibc/configure --prefix=...
gives quite big libraries (libc smaller than libc-2.16, why that?)

../glibc/configure BUILD_CC='gcc' CC='gcc' --prefix=...
gives even bigger libraries, anyone know why?
Is there any special way (configuring) fedora (using F17) gets a small and fast library shipped by default?

../glibc/configure BUILD_CC='gcc' CC='gcc -mx32' CXX='g++ -mx32' --prefix=...
-> make fails
I assume GCC needs a new libgcc to be able to not fail in this case, but trying to build gcc 4.7.1 with multilibs I get errors with incompatible libc!
Can x32 programs use more memory than 32-bit ones? I guess not due to limited pointer size.

[Gareth Jones]

Originally Posted by Raxor

I tried to get the new glibc 2.16 working... make didn't fail, but at make check:
tests/dlfcn: Error 2

[...]

Is there any special way (configuring) fedora (using F17) gets a small and fast library shipped by default?

What are you actually trying to do? If Fedora's C library is genuinely too heavy for your needs, it's probably best to look at a completely different C library like uClibc or Newlib etc.

If you need a system based on a special C library, you either need to start with a system intended for that, or compile one completely from the ground up. Replacing the C library of an existing system with one that has non-trivial differences (either in version or in compile options) is very unlikely to work.

To build a system from scratch, I suggest looking at Linux From Scratch (http://www.linuxfromscratch.org) as a starting point. Fedora is a perfectly good host system for building LFS.

As for differences in file sizes, that could be due to debugging information, or compatibility code – by default, Glibc includes code for compatibility with software compiled against earlier versions, and code for working on older kernels. Both these can be limited or disabled with ./configure options.

../glibc/configure BUILD_CC='gcc' CC='gcc -mx32' CXX='g++ -mx32' --prefix=...
-> make fails
I assume GCC needs a new libgcc to be able to not fail in this case, but trying to build gcc 4.7.1 with multilibs I get errors with incompatible libc!

x86-x32 is a complete third sub-architecture for x86 (in addition to x86-32 and x86-64). This libc won't be compatible with anything currently installed on your system. I'm not sure whether Fedora's kernels are even built with x32 support yet.

Can x32 programs use more memory than 32-bit ones? I guess not due to limited pointer size.

You guess correct. Programs using 32-bit pointers are limited to 4 GiB of memory on a 64-bit kernel, and even less on a 32-bit kernel (the kernel needs some of the 32-bit range for its own use).

[Raxor]

Thanks for your reply. II tried again and copied the output (if needed by anyone).

What are you actually trying to do?

I was curious how this 2.16 release and x32 would affect the size of gcc output files (programs). Even small programs created by gcc are several thousands of bytes big. I also wanted to build static programs. It doesn't seem to be possible with pre-installed libraries. Building statically linked programs should increase both compatibility and speed though I don't know how compatible a static binary really is and how long it will be able to run.

If Fedora's C library is genuinely too heavy for your needs, it's probably best to look at a completely different C library like uClibc or Newlib etc.

Yeah I think I will try them but why should they work when even glibc, the most used one, doesn't?

Replacing the C library of an existing system with one that has non-trivial differences (either in version or in compile options) is very unlikely to work.

Is one version ahead already a too big difference? What parts of the system aren't compatible?

As for differences in file sizes, that could be due to debugging information, or compatibility code

I think I also tried CC='gcc -Os' without success - but will try this again for confirmation. I am aware of some ./configure options but I only see a 'lib' directory in the install directory so I have to use CC= to get 64-bit binaries as well?

x86-x32 is a complete third sub-architecture for x86 (in addition to x86-32 and x86-64). This libc won't be compatible with anything currently installed on your system. I'm not sure whether Fedora's kernels are even built with x32 support yet.

Again: What is incompatible? I don't see any big dependencies that aren't met and why should fedora devs remove x32 support from the kernel as x32 support is the default afaik and removing it will cause incompatibility issues with x32 programs.

You guess correct. Programs using 32-bit pointers are limited to 4 GiB of memory on a 64-bit kernel, and even less on a 32-bit kernel (the kernel needs some of the 32-bit range for its own use).

So x32 doesn't seem to have that shiny future but good to have a confirmation of this.

[Gareth Jones]

Originally Posted by Raxor

I also wanted to build static programs. It doesn't seem to be possible with pre-installed libraries.

You need the static versions of the libraries, provided by packages with names ending -static. For Fedora's Glibc, you'd need to install glibc-static at a minimum (along with glibc-devel for the headers of course).

Building statically linked programs should increase both compatibility and speed though I don't know how compatible a static binary really is and how long it will be able to run.

On 32-bit systems there is a small speed penalty at run-time for DSOs (about 3% on average I think, but I don't have a link for that). On 64-bit the penalty is virtually non-existent as long-mode adds hardware support for relative addressing etc. In both architectures there is load-time overhead due to the dynamic linker resolving symbols, although caching the symbol resolving result with prelink (as Fedora does by default) reduces this. Note that pre-linking increases file sizes due to the cached tables, but doesn't actually use any more memory, as these tables must be generated if absent anyway.

Non-trivial static executables will probably be larger than dynamically-linked versions, unless you use a C library designed to be compact. Glibc is designed for performance (which often conflicts with size), and feature-completeness. Also remember that on Linux, DSOs are shared in memory as well as on disk, so larger static executables are genuinely larger in memory too. Many of the DSO data structures used in dynamic linking can be paged out after loading a program, so the size on disk doesn't necessarily equate to size in memory of the code.

Ultimately the only way to know how these factors will balance out with the programs you're using is to try it of course, but again I'd recommend a C library designed for minimalism if that's your priority. Don't expect them to perform as well as Glibc though, unless you're genuinely short of memory.

Is one version ahead already a too big difference?

It depends on the compile options. Switching sub-architecture (i.e. x32) will change the ABI and absolutely break things. Other things that can break the ABI include optimizations for size and alignment that you could enable, or compatibility code for executables expecting previous versions that you could omit.

Adding CC='gcc' gave apparently worse results - I think I also tried CC='gcc -Os' without success. I am aware of some ./configure options but I only see a 'lib' directory in the install directory so I have to use CC= to get 64-bit binaries as well?

CC is the wrong variable to change compile options. Normally you'd use CFLAGS etc., but Glibc is complex. I suggest looking at the Linux From Scratch instructions for Glibc (and search their mailing lists too). They have detailed instructions for compiling Glibc, and experience as far as optimizations and other unusual compiler options are concerned.

Again: What is incompatible? I don't see any big dependencies that aren't met and why should fedora devs remove x32 support from the kernel as x32 support is the default afaik and removing it will cause incompatibility issues with x32 programs.

It's available in the kernel sources, but I don't know if it's enabled by default when building a kernel, or whether Fedora change that setting. As for compatibility, I'm not aware of any x32 programs in the wild so to speak, or in Fedora's repos. As far as I know, the x32 ABI is still subject to change due to under-using 64-bit numbers (file offsets and time for example), although these may have been resolved now.

So x32 doesn't seem to have that shiny future but good to have a confirmation of this.

Well, my personal opinion is that x32 would have been nice 10 years ago, but now its useful lifetime is already coming to an end except in some specific embedded situations. I wouldn't bother with it unless I had weeks to spend on it and more motivation than I generally have...

[Raxor]

On 32-bit systems there is a small speed penalty at run-time for DSOs (about 3% on average I think, but I don't have a link for that). On 64-bit the penalty is virtually non-existent as long-mode adds hardware support for relative addressing etc. In both architectures there is load-time overhead due to the dynamic linker resolving symbols, although caching the symbol resolving result with prelink (as Fedora does by default) reduces this. Note that pre-linking increases file sizes due to the cached tables, but doesn't actually use any more memory, as these tables must be generated if absent anyway.

Seems that shared objects are very useful for memory saving purposes. Can the difference in startup time even be measured with more or less small programs? Is the 'time' command the most accurate tool? What exactly does 'real', 'sys' and 'user' mean?
If I use a custom library only available on my system and bundle it with the program how does it know where the library actually is?

It depends on the compile options. Switching sub-architecture (i.e. x32) will change the ABI and absolutely break things.

So anyone who needs this new ABI needs to get a new Linux distro/distro update like F18?

CC is the wrong variable to change compile options. Normally you'd use CFLAGS etc., but Glibc is complex. I suggest looking at the Linux From Scratch instructions for Glibc (and search their mailing lists too). They have detailed instructions for compiling Glibc, and experience as far as optimizations and other unusual compiler options are concerned.

I downloaded the LFS book and looked through it. It looks quite complex. Every piece of software needs unique hacks/patches/configure options to work (at least fore this single version). I don't intend to use a system that is bugged and therefore needs patches to work. Also, I won't use this old packages whose bugs should already be fixed in some bug fix version. For example kernel 3.2.6 and glibc 2.14.1. (like half a year old)
But if i can use newer software and don't need that much patches I think I will try it. However I will still need to automate the build process for future updates/enhancements by using a bash script or something.
If I want to reinstall the OS I'll also need to bundle the whole system and/or the software sources and install scripts in a file. I think an executable would be best.. so that I only have to double-click the file and the system installs (lang english, use existing partitioning, root and user passw are asked before install, DE gnome, firefox15, gedit, nautilus, some graphical shell and maybe vlc preinstalled) and get a fully useable system.
For mailing list I need to subscribe :/ (why hidden?) and didn't find any information on glibc optimization except a file 'configparms' for CFLAGS where -march=i486 was set in the LFS book.

[Raxor]

Now LFS 7.2 is out. It contains glibc 2.16.0 and don't need patches for it.
However I still prefer kernel 3.6 over 3.5.2 and everything in the LFS book, especially the complex 'sed' commands, is quite hard to understand for a beginner. I guess this 'sed' don't touch the sources.
Reading the whole book (hundreds of pages) alone would take many hours even without executing the given commands. It would only pay off if I will end up with an easy-to-reinstall bundle which is installable on every modern computer.

[Raxor]

my mistake: the new LFS needs 1 patch for glibc
but can LFS at least be built 64-bit? 32-bit OSes are completely obsolete on hardware that can run 64-bit, aren't they?
Assuming I got a working system (will probably stick with fedora if LFS can only be 32-bit) is it possible to copy it into an executable, a installer (like on windows: double-click it and it installs something) or do I need to create a directory that contains all files and an archive that needs to be extracted first to be useable at all?

[Raxor]

k looks like I'll just stick with Fedora
hopefully the devs will concentrate on a bug-free experience instead of useless facebook integration or other features that will just increase bugs
what about the recent test days? Is a account needed? I couldn't find the link to upload a file (rendercheck).

[stoat]

Originally Posted by Raxor

k looks like I'll just stick with Fedora

That's probably the best idea. Considering what you started this thread talking about, I don't think building an LFS system is what you want. That was suggested to you just as a resource for ideas and guidance for building and installing GLibc.

Anyway, the LFS project by itself is intended mostly to be an educational experience. The system resulting from the book you mentioned is roughly equivalent to Fedora with the "Minimal" package set. It consists of about sixty "packages" which provide the rudimentary underlying structure of an independent bootable Linux system with the most basic core-utils kinds of functions, compiling tools and utilities, and some networking capabilities. There is no X Windows, so nothing graphic, only a tty console with a command prompt for root. No firewall even though the book's steps will result in a wired NIC probably acquiring an IP and connecting during booting. Those kinds of things are part of the BLFS project (Beyond Linux From Scratch). Honestly, there isn't a lot that can be done with an LFS system except to push it on into BLFS. Then it can get useful, competitive, and impressive. But if you were taken aback by the LFS v7.2 book, then the current BLFS book would drive you barking mad.

My BLFS systems are highly prized by me, and there are some good reasons to build one. But I would never start such a project for the reasons in your first post above.



For the record and future readers...



You can build a 64-bit LFS system. See the comments in the section "LFS Target Architectures" in the Preface of the v7.2 book. For now, I still don't value the benefit of a 64-bit system, so I build 32-bit LFS/BLFS systems on my 64-bit hardware. That attitude may not be mainstream and popular, but it is accurate for me.

Your concern about the kernel version in the v7.2 book can be discarded. That v3.5.2 kernel is just what was current when the book was released. It is a triviality to substitute another kernel version.

It's grossly inaccurate to say "every piece of software [in LFS] needs unique hacks/patches/configure options to work". The majority of packages used in LFS/BLFS are not "hacked" and "patched", but are installed as is. And many of the seds merely adjust something trivial like where docs are installed, for example. It's also unfair to say that because the distros all use the same source code and do their own "hacks" and "patches" for the same reasons before rolling them out the door as a binaries. It's no different with them, you just don't see it in the binaries. I don't think you should be concerned about seds and patches. I'm grateful for them. They usually are for correcting the source files for various and inevitable incompatibilities that occur when package versions advance and dependent packages have not caught up (for example and among other reasons). It's an unavoidable thing. Seds and patches fix that stuff.

It's again inaccurate to say that LFS uses "old packages whose bugs should already be fixed". On the contrary, LFS and BLFS are as up-to-date as a distro like Fedora, sometimes ahead (after all, I have glibc-2.16.0 and you don't at the moment). The last three versions of the LFS book (v7.0, v7.1, v7.2) all were released within the last year or so which rivals Fedora's release schedule. The BLFS book has evolved into a sort of rolling release with daily versions. Furthermore, it's naive of you to think the newest software doesn't require patches. Bugs, errors, incompatibilities are a permanent part of all this.

That "double-click" installer daydream of yours made me laugh. But if you can create it, then good. More likely, a BLFS system would be built up in the usual way and distributed or cloned from a tarball. It's how I do it anyway. There have been efforts to automate the LFS build process. I don't think anyone maintains the official automated version of the project nowadays. Some LFS users have attempted to script the process. But the constantly changing package set, and the vastness of the BLFS part of the system, makes automation of the entire build process of a complete distro-comparable system hard to imagine. I can build an LFS system in a single evening. Then I spend the following month of evenings building the BLFS system the way I want it. LFS/BLFS probably is best done the traditional way... one package at a time.



P.S.:



I don't know how it would affect what you were wanting to do, but GLibc-2.16.0 busted stuff all over the place in the LFS and BLFS books. Did you know that? There are numerous packages that required a sed for that. Example...

Originally Posted by LFS v7.2

5.18.1. Installation of Diffutils

Fix an incompatibility between this package and Glibc-2.16.0:

sed -i -e '/gets is a/d' lib/stdio.in.h

I had to apply that or similar seds many times in both books while building my current system. It's happened before though. Last year, libpng-1.5 knocked the slats out of things for a while, too. Anyway, compiling an advanced version of something from source into a working Fedora system might just turn it into a non-working system. Just FYI.

[Raxor]

Originally posted by stoat

That's probably the best idea. Considering what you started this thread talking about, I don't think building an LFS system is what you want. That was suggested to you just as a resource for ideas and guidance for building and installing GLibc.

Thanks for your detailed reply. Currently my main goal is to get Steam running on Fedora 18. When I tried to run it on OS X it was sluggish and tended to hang. Win7-64 recommended me some new settings to start it with and I said 'OK Win7 do what you think is the right thing to do.'. Since then it didn't start anymore and just gave me an error when I tried. Of course with Google or some time I could have solved this prob but I had enough annoyances with different versions of Windows. So I deleted Windows. Probably I will have to install it again becouse of some apps I need (but this time in a VM). If F18 proves to be rock-solid I will use it also for surfing, receiving emails and listening to music (nearly everything aside from firmware updates for my laptop).

However probably I will have to buy an Nvidia card in the future (AMD didn't work for me). My current Intel graphics seem to work but isn't fast. I wanted to upload the rendercheck log file becouse of the Fedora test days and I still should be able to but can't find the upload link.

In my F17 system I spend quite a lot of time just with updating it. I hope this will become faster and it won't cause compatibility problems or other bugs. If updates continue to be quite annoying in F18 maybe I'll disable them or at least kernel updates which can cause (binary) driver incompatibilities (3.6->3.7).
Are the default kernels even useable for a fast, smooth experience or do I need to configure my own kernel for low latency / jitter?

I don't know how it would affect what you were wanting to do, but GLibc-2.16.0 busted stuff all over the place in the LFS and BLFS books. Did you know that? There are numerous packages that required a sed for that. Example...

No, I did not. Thinking of glibc as a standard library I didn't assume it breaks compatibility with other packages.

Anyway, compiling an advanced version of something from source into a working Fedora system might just turn it into a non-working system. Just FYI.

Good to know. If I build new libraries in the future I will certainly keep them seperated from the system. Not to jeopardize a working system has highest priority for me.