I tried:

sudo dd bs=4k if=/dev/mmcblk0 of=/media/1BAB47551C66A42B/raspbian_migs2.gz

It creates an .img file with 7.6 GB (card size, BUT what's on the card has 700 MB).

And:

sudo dd bs=4k if=/dev/mmcblk0 | gzip > /media/1BAB47551C66A42B/raspbian_migs2.gz

creates a .gz file with 2.7 GB.

The original Raspbian (Debian 7 (Wheezy)) from http://www.raspberrypi.org/downloads has 494.44 MiB.

From what's on the SD card, how can I make an image similar in size?

(I'm on Ubuntu.)

up vote 17 down vote accepted

You mention in a comment to RooTer that A) you have reduced the initial partition size with gparted, but dd still copies the whole card, and B) that you want to include both partitions in the image.

Issue "A" is easy to explain: you are still copying the whole card because that's what /dev/mmcblk0 refers to. The individual partitions are of course /dev/mmcblk0p1 and /dev/mmcblk0p2. This is the complication in issue "B", but you cannot simply dd each partition and concatenate the two files together, because of the partition table at the beginning of /dev/mmcblk0 which indexes the beginning and length of each partition. Without that, the image will be unusable.

However, you can get the length of each partition from fdisk -l, and use that to determine some parameters for dd. For example:

> fdisk -l /dev/mmcblk0

Disk /dev/mmcblk0: 16.1 GB, 16138633216 bytes
4 heads, 16 sectors/track, 492512 cylinders, total 31520768 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00017b69

Device Boot      Start         End      Blocks   Id  System
/dev/mmcblk0p1    8192      122879       57344    c  W95 FAT32 (LBA)
/dev/mmcblk0p2  122880    26746879    13312000   83  Linux

The "Start" and "End" units are sectors, and notice the sector size is given, 512 bytes. For /dev/mmcblk0p2, 26746879 (the last sector) - 122880 (the first sector) = 26623999 / 2 (for 2 sectors per kB) / 1024 (kB per MB) / 1024 (MB per GB) = 12.69, which I grew the partition using gparted to 12 GB, so this looks correct (really I should be using 1000 and not 1024 as the divisor with storage, which works out to 13.31 GB, but I suspect gparted and some other tools also use 1024).

So the first thing you want to check is that your second partition is really the smaller size that you set it to. Next, just use those numbers with dd; for me it would be:

dd if=/dev/mmcblk0 of=rpi.img bs=512 count=26746880

I've got an extra sector there to avoid any kind of off by one misunderstanding of how dd works. There's a simple way to check if this worked:

> fdisk -l rpi.img

Disk rpi.img: 102 MB, 102400000 bytes
255 heads, 63 sectors/track, 12 cylinders, total 200000 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00017b69

Device Boot      Start         End      Blocks   Id  System
rpi.img1          8192      122879       57344    c  W95 FAT32 (LBA)
rpi.img2        122880    26746879    13312000   83  Linux

Notice there is a bit of a discrepancy here: the "Start" and "End" sectors match the original partition table, but the total size in the stats up top is only 102 MB! This is because I actually used count=200000 as the param to dd because I did not really want to bother with a 12 GB copy (notice also "total 200000 sectors"). The reason the table at the bottom does not reflect this is because fdisk is getting its information from the partition data copied verbatim at the beginning of the image from the beginning of the SD card, which, as I mentioned in the second paragraph, is vital to maintain. If I had (properly) copied the rest, the numbers would be copacetic and the image would be viable.

Give that a try. :)

  • On OSX, fdisk doesn't clearly print out the sector size in bytes. It offers instead "geometry: 966/255/63 [15523840 sectors]" representing cylinders/heads/sectors. What values of bs and count should be used in this case? – Arthur Hebert Jan 21 '15 at 5:06
  • @ArthurHebert: Total bytes / total sectors. E.g. in the first case above it would be 16138633216 / 31520768 = 512, in the second 102400000 / 200000 = 512. – goldilocks Jan 21 '15 at 14:22
  • 1
    You can use fdisk -l <device> and that should print the table without going into interactive mode. – berto Oct 31 '15 at 2:06

I would guess problem lies in once used sectors that still have dirt in them. Once file is deleted, only metadata is removed from filesystem, not the data itself, thus leaving some random one-zeros instead of easy to compress zero-only blocks.

Easy solution, but requires rewriting all free space on the card. Remember that SD card lifespawn is limited by number of rewrites, so this is not preferred method.

dd bs=4M if=/dev/zero of=/root/junk
sync
rm junk

More involved solution, as you need to install zerofree on another computer which won't be using that SD card at the time.

zerofree /dev/mmcblk0p2

For more read up on http://intgat.tigress.co.uk/rmy/uml/index.html

What you have to remember is that doing dd of /dev/mmcblk0 you copy whole device, even if partitions are smaller. If you used raspi-config to expand main partition before doing one of above methods you will be fine.

P.S. If you don't mind changing format image file you can use partimage which for known filesystems omits freed blocks even if they still have some dirt in them. Again partimage is best to be used when filesystem is not mounted to avoid corruption of backup. You probably could get away with remounting it read-only, but I will leave it to your discretion.

  • as part of my quest on trying to get this done, i used gparted to reduce the sdcard partition to acomodate only the data i have, then i tried dding and the results are the same 7.6GB file, partimage cant save 2 partitions (/boot + /) into 1 image – mf_ Jul 6 '13 at 21:56
  • maybe I didn't make it clear enough - it should make a difference when you compress it, as you have tried with gzip before. – RooTer Jul 7 '13 at 21:10
  • filling the card in this manner will cause a whole bunch of write cycles and will shorten the lifespan of the card. try dd bs=4M if=/dev/zero of=/root/junk – nc4pk Jul 8 '13 at 2:35
  • @tapped-out thx, edited;) – RooTer Jul 11 '13 at 16:41

Short answer -- use 2GB SD card.

Long answer, dd has no idea where the "good" data ends, you have to tell it somehow.

There are two ways, easiest is to use 2GB SD card, that will automatically stop copying beyond 2GB and will result in 500MB compressed file as you desire.

The other way, more complicated an involved one, is to calculate the correct data size from your partition table and specify this correct size as a parameters to dd command. You may use bs=XXX (block size) and count=XXX (block count) parameters for that purpose. For example, you may specify bs=10M for 10MB block size (that would definitely make copying much faster compared to 4k block size you use in your commands) and count=200 to copy 10MB * 200 = 2000MB (2GB). You may need to adjust block size and block count according to your SD card partition scheme.

  • 1
    Giving dd a specific size will NEVER EVER work. That presumes all of the actual data in the filesystem is neatly arranged at the beginning of the device, so if you have 2GB on an 8 GB partition you just need to copy the first 2GB. This is FALSE. Those 2 GB of data will be scattered across the entire space, especially on modern SD cards which do not reuse blocks twice until all available blocks have been used at least once (this is called wear leveling and it prolongs the life of the card). – goldilocks Jul 7 '13 at 12:14
  • @goldilocks what if i do resize the sdcard in order to shrink all partitions to the maximum possible size(data only) ?? – mf_ Jul 7 '13 at 21:07
  • @goldilocks please, read the question and the answer more carefully, i'm talking about 2GB partition on a 8GB SD card, not about 2GB data on 8GB partition, as your wild imagination somehow told you. – lenik Jul 7 '13 at 23:50
  • 1
    lenik : Yeah I'm a little stunned I interpreted you that way, all apologies -- unfortunately I can't reverse my downvote unless you edit the post :/ although I still don't think this answer is particularly useful (no offence -- because this is not simply dd'ing a partition, instructions on how to do such are, again, not useful), but I'll do that. @mf_ Yes, that's doable (did you read my answer? It will work...) – goldilocks Jul 9 '13 at 15:49
  • @goldilocks I've edited the answer. Don't know why it's not useful, especially when you gave exact same answer, just with more details. – lenik Jul 10 '13 at 1:14

dd - copy and convert is not the right tool to do the job you have requested. It is a low level, sector by sector copy (and convert) tool, which is great to copy boot sectors, format devices, and all sorts of low level tasks. When you use dd you are copying sector-by-sector to the image, even if it is not included in the File System Structure.

The images provided by the Raspberry Pi foundation, are specially compiled images with install scripts, unzipping binaries and initial setup, after which you have to get updates from the internet anyway - Which is all deliberate but quite a task to get it workin like this.

One popular solution to avoid copying empty sectors is to use a File Level copy system - and CloneZilla is standalone, bootable from a CD, similar to ye olde Norton Ghost but clonezilla supports the Linux (and more) file systems. So, it will only copy files that are in use and create a container from those files only. Reduces the size significantly!

I had this same exact question and wanted an easy to use tool. After searching and not finding one I wrote mkimg.sh. I outline the process I used at: https://raspberrypi.stackexchange.com/a/37899/32585

Your Answer

 
discard

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.