Some Questions About Building RPMs

[Hlingler]

Hello:

A lot of the stuff I build comes from CVS, SVN, or GIT repositories. Regardless of the type of source code repository, I have created a central folder (/home/shared/Source_Code/) with sub-folders for all this source code. A lot of this stuff gets updated fairly often, and the sizes tend to be quite large. Up until now, I've had very good luck using 'checkinstall' to build RPMs from all this source code. But the checkinstall app has limitations - and one nasty little bug - and the checkinstall code itself hasn't been updated in over a year, and I'm starting to see alarming things happen when I use it. Like: files from the real packages on the real filesystem get erased during a checkinstall run. Not good. Very annoying. Time to learn how to make real RPMs....

Problem: I've been reading up on RPM and how to build RPMs with it. I've done this a few times before with pre-packaged SRPMS and such - fairly simple builds, just put the stuff where it belongs and pull the trigger. But, it seems to require a specific filesystem format - i.e., the five folders BUILD, RPMS, SOURCES, SPECS, SRPMS. The question is: how do I set up a build environment so that I can use all the various existing source code folders as the SOURCE folder(s), and not have to move massive amounts of data around, shifting and clearing out the default /usr/src/redhat/SOURCE/ folder (and/or others) each time I need to build an RPM? In other words, I'd like to keep the existing CVS, SVN and GIT source code folders where they are, and tell RPM to use one or another particular folder as the SOURCE folder, depending on what I need to build at the moment. This seems to me like a rather simple question, but I have no idea how to do this, and I haven't found anything to guide me. Advice/help on this will be most appreciated.

Thanx,
V

[daneel971]

Originally Posted by Hlingler

A lot of the stuff I build comes from CVS, SVN, or GIT repositories.

Me too. Usually, the multimedia stuff (mplayer, x264, ffmpeg, libdvdnav/libdvdread). I'll be more than happy to share the specfiles, if you want.

Originally Posted by Hlingler

(rpmbuild) it seems to require a specific filesystem format - i.e., the five folders BUILD, RPMS, SOURCES, SPECS, SRPMS.

Right. The rpmdevtools package has all the scripts required to set up you environment, for buildind rpms as a user. Then you can write your .rpmrc and .rpmmacros, for fine tuning your build options.
(BTW: F9 seems to ignore my .rpmrc file...)

Originally Posted by Hlingler

The question is: how do I set up a build environment so that I can use all the various existing source code folders as the SOURCE folder(s), and not have to move massive amounts of data around, shifting and clearing out the default /usr/src/redhat/SOURCE/ folder (and/or others) each time I need to build an RPM?
I have no idea how to do this, and I haven't found anything to guide me.

AFAIK, simply, it can't be done.
Recently I started using Archlinux: the package building is managed by "makepkg": this program, using a PKGBUILD file (very similar to rpms spec file: after a couple of days, I was already building my packages...) it can download the sources (and the patches, if needed) directly, starting even with a git/svn/cvs etc. checkout - you can keep the source trees after the building and update them or you can delete them and start everytime with a new checkout.

[Hlingler]

Hi daneel971,

And thanks for the response. I temporarily evaded the problem by making all files/folders in /usr/lib/ and /usr/include immutable (chattr -R +i /usr/lib/) while running checkinstall. I thought this would probably cause checkinstall to fail and abort, but it didn't - the packages were built successfully.

AFAIK, simply, it can be done.

So, you're saying that it can be done? Or cannot? If it can be done, then how? Thanks.

Ciao,
V

[daneel971]

Writing while you're sleeping is bad. I should remember that...
I meant it CAN'T be done.
Anyway, if you start editing existing specfiles, writing from scratch yours will be relatively simple. I have always found very useful the "Maximum RPM" guide and the pages on the fedora wiki.
As for building from git/svn, etc, I simply make a tarball of a given snapshot and then clean the build directories (this can be done automatically with rpmbuild).

[Hlingler]

Thanks again, daneel971. Yes, I've been reading MaximumRPM. I like your suggestion about making tarball of snapshot and using that to build an RPM.

Thanx Again for your time and help and suggestions - I'm off to build some RPMs!

V

[jbkt23]

I'm bringing this thread to the fore again, as the original posters seem to have knowledge of .spec files for building from source. I'm posting because I am trying to build the kmod-madwifi driver from source and I am having a couple of difficulties.
The first is that if I compile against the kerne-devel files for the kernel I am needing the driver for there is some error that causes the compile to fail.
The second is if I compile against the running kernel that doesn't need the driver the module files are not optimized for size. They are sometimes 10 times the size of the same module .ko that is installed via the update process.
Typically when I build from source, and I have been successful in building nvidia drivers for custom kernels, I install the source and then run "rpmbuild -bp --target <uname -m> package.spec" to create the build directories. I do not like doing build installs because they sometimes put modules in unexpected places. I also have edited the .spec files to build for a specific kernel version, but I find the madwifi spec file comments to be a bit ambiguous.
Some more background. To install the driver source I had to grab a bunch of dependencies all for the prior kernel version since livna has not released the versions for the latest kernel, had they, I wouldn't have embarked upon this adventure.
So I build the binaries via the make command in the top level source directory "make KERNELPATH=/usr/src/kernels/<targetkernel.version> " as a normal user. Then I would as root manually copy the files to the modules directory locations, reboot, and then run depmod and modprobe to enable the modules. This has worked well in the past for kernel dependent modules since they are not updated. I would use rpmbuild to make the binary package for non kernel dependent packages.
What is the question you ask? Well, I guess how do I know that I've prepared the package source properly for the kernel I want to build against given that the only available source is for a prior kernel version? What do I edit in the spec file or issue on the rpmbuild command line to effect the proper source prep?
I have read the man file for rpmbuild and also have perused a tutorial that was posted here a while back. One of the issues is that if your not a programmer, like myself, some of the terms are hard to follow or seem ambiguous as I mentioned before.

enough for now

[Hlingler]

I don't understand the question(s).

First, *.spec files only apply to 'rpmbuild' using SRPMs (*.src.rpm), not raw "source code" per se.

Regardless of how you build (via 'rpmbuild ...' or 'make ...'), you can optimize the build by preceding the command with, for example:
[Vince@localhost ~]$ CFLAGS="-O3 -pipe -m32 -march=i386 -mtune=generic" CXXFLAGS=$CFLAGS [./configure OR rpmbuild OR make] ...

Without passing such flags to the compiler, defaults will be used (usually: "-O -g", which means low optimization and include debugging krap, guess or omit the rest). This usually results in much larger executables, mainly due to the debugging stuff.

The depmod command can and should be run before re-booting to the new kernel, for example:
depmod -a 2.6.26.3-29.fc9.i686

Without appending a kernel number, the running kernel is assumed (`uname -r`).

If needed, you can enforce the desired module to be loaded by simply placing the command into /etc/rc.d/rc.local:
modprobe <insert-module-name>

I know of no "kernel prep" whatsoever that needs to be done for any typical rpmbuild.

Hope that helps...
V

[leigh123linux]

Originally Posted by jbkt23

I'm bringing this thread to the fore again, as the original posters seem to have knowledge of .spec files for building from source. I'm posting because I am trying to build the kmod-madwifi driver from source and I am having a couple of difficulties.
The first is that if I compile against the kerne-devel files for the kernel I am needing the driver for there is some error that causes the compile to fail.
The second is if I compile against the running kernel that doesn't need the driver the module files are not optimized for size. They are sometimes 10 times the size of the same module .ko that is installed via the update process.
Typically when I build from source, and I have been successful in building nvidia drivers for custom kernels, I install the source and then run "rpmbuild -bp --target <uname -m> package.spec" to create the build directories. I do not like doing build installs because they sometimes put modules in unexpected places. I also have edited the .spec files to build for a specific kernel version, but I find the madwifi spec file comments to be a bit ambiguous.
Some more background. To install the driver source I had to grab a bunch of dependencies all for the prior kernel version since livna has not released the versions for the latest kernel, had they, I wouldn't have embarked upon this adventure.
So I build the binaries via the make command in the top level source directory "make KERNELPATH=/usr/src/kernels/<targetkernel.version> " as a normal user. Then I would as root manually copy the files to the modules directory locations, reboot, and then run depmod and modprobe to enable the modules. This has worked well in the past for kernel dependent modules since they are not updated. I would use rpmbuild to make the binary package for non kernel dependent packages.
What is the question you ask? Well, I guess how do I know that I've prepared the package source properly for the kernel I want to build against given that the only available source is for a prior kernel version? What do I edit in the spec file or issue on the rpmbuild command line to effect the proper source prep?
I have read the man file for rpmbuild and also have perused a tutorial that was posted here a while back. One of the issues is that if your not a programmer, like myself, some of the terms are hard to follow or seem ambiguous as I mentioned before.

enough for now

Try post #18 ( it will work with any kmod )

http://forums.fedoraforum.org/forum/...8&page=1&pp=20

[jbkt23]

Hlingler thanks for replying,

Yes, I was working with the srpm.

In the compile options statement:

CFLAGS="-O3 -pipe -m32 -march=i386 -mtune=generic" CXXFLAGS=$CFLAGS [./configure OR rpmbuild OR make] ...

-m32 would this be -m64 for x86_64
-march=i386 is this generic for I586 and i686? and would it be imarch=x86_64 for 64bit?

So the rpmbuild and its options or make and its options comes last after the flags statements?

I usually compile for the running kernel so I had not known that you could depmod -a for another installed kernel.

The "prep" I was referring to is whatever the <package.spec> file does to the srpm sources, whether it is apply a patch and or create the source tree for a specific architecture. When I compiled the nvidia driver for my custom kernel I edited the spec file kversion line to match the kernel I was compiling for. See the link below for a better explanation.

http://fedoraforum.org/forum/showthread.php?t=168033

[jbkt23]

Leigh123,
I will give the instructions in the link a try but a quick question. The target of the rpmbuild --rebuild command in the link is a package.src.rpm. So is that implying that I just go to the directory the source was downloaded to and issue it?

And, then the binary rpm will be created and placed in the usual place "../../src/RPMS/ ?

[leigh123linux]

Originally Posted by jbkt23

Leigh123,
I will give the instructions in the link a try but a quick question. The target of the rpmbuild --rebuild command in the link is a package.src.rpm. So is that implying that I just go to the directory the source was downloaded to and issue it?

And, then the binary rpm will be created and placed in the usual place "../../src/RPMS/ ?

Yes that's correct .

su
yum groupinstall "Development Tools"
yum install rpmdevtools yum-utils kmodtool
exit
rpmdev-setuptree
yumdownloader --source kmod-ndiswrapper
rpmbuild --rebuild --target=i686 --define "kernels 2.6.26.3-29.fc9.i686" ndiswrapper-kmod*.src.rpm
su -c  "rpm -ihv ~/rpmbuild/RPMS/i686/kmod-ndiswrapper*.rpm"

build the package as non root , the binary package will be in /home/*/rpmbuild/RPMS/i686/

[Hlingler]

Note that it's including the debugging stuff mainly ("-g") and also partly optimization level ("-O[1-2-3]") that's really the size-killer for those executables/libraries.

Originally Posted by jbkt23

Hlingler thanks for replying,

Yes, I was working with the srpm.

In the compile options statement:

CFLAGS="-O3 -pipe -m32 -march=i386 -mtune=generic" CXXFLAGS=$CFLAGS [./configure OR rpmbuild OR make] ...

-m32 would this be -m64 for x86_64

Yes, I believe that is correct.

-march=i386 is this generic for I586 and i686?

Yes, correct: generic 32-bit. Not sure what to put in that place for 64-bit. Maybe "-march=athlon"??? Safe to omit.

and would it be imarch=x86_64 for 64bit?

I'm not sure on this one - may need to search around to verify. See previous answer. Safe to omit.

So the rpmbuild and its options or make and its options comes last after the flags statements?

Yes, correct: either preceed the command with the variable declarations as I showed, or set with the 'export' command.

I usually compile for the running kernel so I had not known that you could depmod -a for another installed kernel.

Much handier to depmod before the re-boot....

The "prep" I was referring to is whatever the <package.spec> file does to the srpm sources, whether it is apply a patch and or create the source tree for a specific architecture. When I compiled the nvidia driver for my custom kernel I edited the spec file kversion line to match the kernel I was compiling for. See the link below for a better explanation.

http://fedoraforum.org/forum/showthread.php?t=168033

Patches are often applied and the OS particulars are gathered, such as where the various *-devel files are located, but nothing is done to "prep" the kernel. There is a way I'm sure to pass the kernel number to the build, but not sure of the details....

HTH,
V

[jbkt23]

I need to tell rpmbuild where the kernel-devel-2.6.26.3-14.fc8-i686 headers are. It is not the running kernel.

$ rpmbuild --rebuild --target=i686 --define "kernels 2.6.26.3-14.fc8-i686" madwifi-kmod-0.9.4-15.lvn8.src.rpm
............
..........
Building for target i686
error: Failed build dependencies:
kernel-devel-i686 = 2.6.26.3-14.fc8-i686 is needed by madwifi-kmod-0.9.4-15.fc8.i686

[Hlingler]

Well then, install kernel-devel-2.6.26.3-14.fc8.i686, then put all of the above together:
CFLAGS="-O3 -pipe -m32 -march=i386 -mtune=generic" CXXFLAGS=$CFLAGS rpmbuild --rebuild --target=i686 --define "kernels 2.6.26.3-14.fc8.i686" madwifi-kmod-0.9.4-15.lvn8.src.rpm

NOTE that the command is to be typed on ONE SINGLE LINE.

Or the equivalent command with x86_64 variables substituted instead.

V

P.S. You sure that's not a typo:
--define "kernels 2.6.26.3-14.fc8-i686"

should be:
--define "kernels 2.6.26.3-14.fc8.i686"

[jbkt23]

Originally Posted by Hlingler

Well then, install kernel-devel-2.6.26.3-14.fc8.i686, then put all of the above together:
CFLAGS="-O3 -pipe -m32 -march=i386 -mtune=generic" CXXFLAGS=$CFLAGS rpmbuild --rebuild --target=i686 --define "kernels 2.6.26.3-14.fc8.i686" madwifi-kmod-0.9.4-15.lvn8.src.rpm

NOTE that the command is to be typed on ONE SINGLE LINE.

Or the equivalent command with x86_64 variables substituted instead.

V

P.S. You sure that's not a typo:
--define "kernels 2.6.26.3-14.fc8-i686"

should be:
--define "kernels 2.6.26.3-14.fc8.i686"

The kernel-devel package "/usr/src/kernels/2.6.26.3-14.fc8-i686" is installed and yes there is a typo in the line I posted because I issued the command both ways just to see if I had to match the kernel-devel directory heading.