Fedora Linux Support Community & Resources Center
  #1  
Old 13th May 2016, 03:45 PM
lsatenstein Offline
Registered User
 
Join Date: Jun 2005
Location: Montreal, Que, Canada
Posts: 3,658
linuxfedorafirefox
Formatting an external backup drive, xfs, btrfs,ext4 or f2fs

I have inherited a third backup drive (225gigs). I also owned a 1 terrabyte drive that I partitioned as 50% ntfs and 50% ext4. I want to format this 225gig drive for LInux. Instinctively I thought of using ext4.


I never thought about it before, but it came to mind that perhaps, in lieu of ext4, I should use xfs, btrfs or the newer f2fs formats. I had f2fs on my desktop as my home directory for almost 6 months. I never had a crash, and never saw any performance issues.

With btrfs, I had some problems, and so I am shy to use that filesystem.

What I am thinking of doing is formatting the drive ext4, and looking at disk utilisation, then do likewise with xfs and f2fs.

In general, I think that the differences between the three mentioned should be less than 50 megs (meta data use).

Phoronix posted an article about this topic, but did not include meta data (control information sizes). f2fs did quite well on retrievals.
Linux 4.3 File-System Comparison With Btrfs, EXT4, XFS, F2FS http://www.phoronix.com/scan.php?pag...x-43-ssd&num=1


Your comment please.
__________________

Leslie
in Montreal


Interesting web sites list
http://forums.fedoraforum.org/showth...40#post1697840

Last edited by lsatenstein; 13th May 2016 at 04:43 PM.
Reply With Quote
  #2  
Old 14th May 2016, 06:51 AM
stevea
Guest
 
Posts: n/a
linuxcroschrome
Re: Formatting an external backup drive, xfs, btrfs,ext4 or f2fs

It's a very meaningless comparison -- apples to antelope.

The first question that should occur is 'how do you intend to measure this overhead' ?
There are no general tools for that task, and AFAIK most FS tools don't report it.

Note that much, but not all of an in-use file-system overhead is 'metadata' or 'data'. There are one or more header blocks describing the FS, and there are tables - often bit-tables - of the blocks in use, then there are journals, and likely some structures to optimize the block allocation/deallocation - those aren't metadata.

Data - The size of the file data isn't particularly variable - except that btrfs and perhaps others, support data compression and this has a very positive effect under many workloads. Some, and again certainly btrfs, allows data duplication raid-like ways.

Metadata - the on-disk inode and attribute data - again doesn't vary much between file systems, except that some, aka btrfs defaults to meta-data duplication (raid0) on a single volume.

The rest varies A LOT! Especially note that by default ext[34] and xfs create journals, and allocate 5% of the file-system space as 'reserve' or 'over-provision'. Also ext* creates fixed basic inodes for all files as fs creation ! Most of the others create disk inodes dynamically (see /etc/mfe2fs.conf) . An ext4 inode_ratio of 16k and inode size of 256 means you will pre-allocate 1.5625% of space to inodes. So you are immediately talking about the default carve-out of ~3.5GB for inodes and ~11.25GB as reserved space !! That's a LOT Lot more than 50mb diff from btrfs - maybe 2+ orders of magnitude difference.

--

F2FS - Using f2fs on a rotating drive is ridiculous. This fs has a lot of features that only add value on an SSD or flash drive, and these do add to space overhead. How much - I don't know. But it takes a very special person to want wear-leveling or hot/warm/cold file segregation or background cleaning on a rotating disk; perhaps the same sort of person who want's ketchup on chantarelles or mixes cognac with kool-aid.

Btrfs - when creating the FS you are immediately faced with the decision to duplicate (or form more complex raid models) meta-data and/or data separately. The default is to 'dup' the metadata (raid0) and not the data on a single drive. So you are already buying 2x the inode overhead *** but *** you get certain features and value for that expenditure of space. No 5% tax! Despite your bad experience and failure to use the appropriate tools to recover - this is a stable, high feature, good performance FS.

Nilfs - should be included in any reasonable comparison.

---


So aside from ....
a/ can't be practically measured,
b/ measures nothing useful,
c/ avoids the obvious winner based on a spurious personal bias
d/ fails to test important contenders.
well you're batting 100 or is lacrosse the intention ?
Reply With Quote
  #3  
Old 14th May 2016, 08:00 AM
flyingdutchman Online
Registered User
 
Join Date: Jan 2015
Location: Al Ain, UAE
Posts: 522
macosfirefox
Re: Formatting an external backup drive, xfs, btrfs,ext4 or f2fs

Well, you want to store backups. Therefore, I would recommend btrfs or openzfs, since they can do file and block level deduplication and save oodles of space.
__________________
--
Have fun!
http://www.aeronetworks.ca
Reply With Quote
  #4  
Old 15th May 2016, 12:35 AM
lsatenstein Offline
Registered User
 
Join Date: Jun 2005
Location: Montreal, Que, Canada
Posts: 3,658
linuxfedorafirefox
Re: Formatting an external backup drive, xfs, btrfs,ext4 or f2fs

I took the btrfs route, Thank you all!
__________________

Leslie
in Montreal


Interesting web sites list
http://forums.fedoraforum.org/showth...40#post1697840
Reply With Quote
  #5  
Old 15th May 2016, 06:29 AM
stevea
Guest
 
Posts: n/a
linuxcroschrome
Re: Formatting an external backup drive, xfs, btrfs,ext4 or f2fs

Quote:
Originally Posted by flyingdutchman View Post
Well, you want to store backups. Therefore, I would recommend btrfs or openzfs, since they can do file and block level deduplication and save oodles of space.
Excellent thought - duperemove appears to not be available in any package - has to be built from source.

You can also do btrfs lzo or zlib compression (similar compression for openzfs) and that is often a big win for both speed and space (assuming the cpu has idle time and space is a concern).

LVM, nilfs, btrfs or openzfs on the source filesystems would allow snapshotting for a proper backup.
Reply With Quote
  #6  
Old 15th May 2016, 08:35 AM
flyingdutchman Online
Registered User
 
Join Date: Jan 2015
Location: Al Ain, UAE
Posts: 522
linuxfedorafirefox
Re: Formatting an external backup drive, xfs, btrfs,ext4 or f2fs

Since you chose btrfs, see this:
https://btrfs.wiki.kernel.org/index.php/Deduplication
__________________
--
Have fun!
http://www.aeronetworks.ca
Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Benchmark ext4 vs f2fs lsatenstein Linux Chat 13 28th March 2016 06:13 PM
Does a BTRFS Drive Need a BTRFS Backup Drive? joncr Using Fedora 5 15th February 2014 09:32 PM
formatting or partitioning an external drive moosepig Linux Chat 5 26th August 2010 01:34 AM
Formatting External Drive For Use between Mac, Fedora, & V_sta Library Eye Mac Chat 5 24th April 2009 12:23 AM
error while formatting external usb hard drive daviddoria Hardware & Laptops 0 12th November 2008 06:17 PM


Current GMT-time: 11:59 (Friday, 20-01-2017)

TopSubscribe to XML RSS for all Threads in all ForumsFedoraForumDotOrg Archive
logo

All trademarks, and forum posts in this site are property of their respective owner(s).
FedoraForum.org is privately owned and is not directly sponsored by the Fedora Project or Red Hat, Inc.

Privacy Policy | Term of Use | Posting Guidelines | Archive | Contact Us | Founding Members

Powered by vBulletin® Copyright ©2000 - 2012, vBulletin Solutions, Inc.

FedoraForum is Powered by RedHat