Issue #123: Errors with NVMe on Compute Module 4. #147

Open
geerlingguy wants to merge 1 commits from geerlingguy/123-nvme into master
geerlingguy commented 2022-09-22 00:41:10 +02:00 (Migrated from github.com)

Fixes #123. Allows users to work with NVMe drives on the CM4.

Fixes #123. Allows users to work with NVMe drives on the CM4.
mike-pisman (Migrated from github.com) approved these changes 2022-09-22 22:19:25 +02:00
mike-pisman (Migrated from github.com) left a comment

Looks good to me, thank you!

Looks good to me, thank you!
framps (Migrated from github.com) reviewed 2022-09-23 09:46:36 +02:00
@@ -577,3 +576,4 @@
if [[ $disk == *"mmcblk"* || $disk == *"nvme"* ]]
then
SD_slot_boot=1
disk=${disk:0:7}
framps (Migrated from github.com) commented 2022-09-23 09:46:36 +02:00

Just a comment: I first thought this line will fail for nvme but it works fine 👍 . It works for nvme only as long as for /dev/nvme<x>n<y> x and y has only one digit. But that's a restriction also on the previous code which extracts the partition number.

Just a comment: I first thought this line will fail for nvme but it works fine :+1: . It works for nvme only as long as for /dev/nvme\<x>n\<y> x and y has only one digit. But that's a restriction also on the previous code which extracts the partition number.
geerlingguy commented 2022-09-26 23:38:36 +02:00 (Migrated from github.com)

Testing it on one of my Pis with eMMC boot and an NVMe drive:

pi@dvr:~/rpi-clone $ lsblk
NAME         MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
mmcblk0      179:0    0 14.6G  0 disk 
├─mmcblk0p1  179:1    0  256M  0 part /boot
└─mmcblk0p2  179:2    0 14.3G  0 part /
mmcblk0boot0 179:32   0    4M  1 disk 
mmcblk0boot1 179:64   0    4M  1 disk 
nvme0n1      259:0    0  7.3T  0 disk 
├─nvme0n1p1  259:1    0  600M  0 part 
├─nvme0n1p2  259:2    0    1G  0 part 
└─nvme0n1p3  259:3    0  7.3T  0 part 

pi@dvr:~/rpi-clone $ sudo rpi-clone nvme0n1
Error: /dev/nvme0n1: unrecognised disk label

Booted disk: mmcblk0 15.6GB                Destination disk: nvme0n1 8.0TB
---------------------------------------------------------------------------
Part      Size    FS     Label           Part   Size  FS  Label  
1 /boot   256.0M  fat32  --                                      
2 root     14.3G  ext4   rootfs                                  
---------------------------------------------------------------------------
== Initialize: IMAGE partition table - partition number mismatch: 2 -> 0 ==
1 /boot               (31.5M used)   : MKFS  SYNC to nvme0n1p1
2 root                (1.4G used)    : RESIZE  MKFS  SYNC to nvme0n1p2
---------------------------------------------------------------------------
Run setup script       : no.
Verbose mode           : no.
-----------------------:
** WARNING **          : All destination disk nvme0n1 data will be overwritten!
-----------------------:

Initialize and clone to the destination disk nvme0n1?  (yes/no): yes
Optional destination ext type file system label (16 chars max): 

Initializing
  Imaging past partition 1 start.
  => dd if=/dev/mmcblk0 of=/dev/nvme0n1 bs=1M count=8 ...
  Resizing destination disk last partition ...
    Resize success.
  Changing destination Disk ID ...The size of this disk is 7.3 TiB (8001563222016 bytes). DOS partition table format cannot be used on drives for volumes larger than 2199023255040 bytes for 512-byte sectors. Use GUID partition table format (GPT).

  => mkfs -t vfat -F 32  /dev/nvme0n1p1 ...
  => mkfs -t ext4  /dev/nvme0n1p2 ...

Syncing file systems (can take a long time)
Syncing mounted partitions:
  Mounting /dev/nvme0n1p2 on /mnt/clone
  => rsync // /mnt/clone with-root-excludes ...
  Mounting /dev/nvme0n1p1 on /mnt/clone/boot
  => rsync /boot/ /mnt/clone/boot  ...

Editing /mnt/clone/boot/cmdline.txt PARTUUID to use db6793cc
Editing /mnt/clone/etc/fstab PARTUUID to use db6793cc
===============================
Done with clone to /dev/nvme0n1
   Start - 16:36:11    End - 16:36:54    Elapsed Time - 0:43

Cloned partitions are mounted on /mnt/clone for inspection or customizing. 

Hit Enter when ready to unmount the /dev/nvme0n1 partitions ...
  unmounting /mnt/clone/boot
  unmounting /mnt/clone
===============================

pi@dvr:~/rpi-clone $ lsblk
NAME         MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
mmcblk0      179:0    0 14.6G  0 disk 
├─mmcblk0p1  179:1    0  256M  0 part /boot
└─mmcblk0p2  179:2    0 14.3G  0 part /
mmcblk0boot0 179:32   0    4M  1 disk 
mmcblk0boot1 179:64   0    4M  1 disk 
nvme0n1      259:0    0  7.3T  0 disk 
├─nvme0n1p1  259:1    0  256M  0 part 
└─nvme0n1p2  259:2    0    2T  0 part 

Note that I was testing with an 8 TB NVMe drive, and it was only sized to 2 TB...

Testing it on one of my Pis with eMMC boot and an NVMe drive: ``` pi@dvr:~/rpi-clone $ lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT mmcblk0 179:0 0 14.6G 0 disk ├─mmcblk0p1 179:1 0 256M 0 part /boot └─mmcblk0p2 179:2 0 14.3G 0 part / mmcblk0boot0 179:32 0 4M 1 disk mmcblk0boot1 179:64 0 4M 1 disk nvme0n1 259:0 0 7.3T 0 disk ├─nvme0n1p1 259:1 0 600M 0 part ├─nvme0n1p2 259:2 0 1G 0 part └─nvme0n1p3 259:3 0 7.3T 0 part pi@dvr:~/rpi-clone $ sudo rpi-clone nvme0n1 Error: /dev/nvme0n1: unrecognised disk label Booted disk: mmcblk0 15.6GB Destination disk: nvme0n1 8.0TB --------------------------------------------------------------------------- Part Size FS Label Part Size FS Label 1 /boot 256.0M fat32 -- 2 root 14.3G ext4 rootfs --------------------------------------------------------------------------- == Initialize: IMAGE partition table - partition number mismatch: 2 -> 0 == 1 /boot (31.5M used) : MKFS SYNC to nvme0n1p1 2 root (1.4G used) : RESIZE MKFS SYNC to nvme0n1p2 --------------------------------------------------------------------------- Run setup script : no. Verbose mode : no. -----------------------: ** WARNING ** : All destination disk nvme0n1 data will be overwritten! -----------------------: Initialize and clone to the destination disk nvme0n1? (yes/no): yes Optional destination ext type file system label (16 chars max): Initializing Imaging past partition 1 start. => dd if=/dev/mmcblk0 of=/dev/nvme0n1 bs=1M count=8 ... Resizing destination disk last partition ... Resize success. Changing destination Disk ID ...The size of this disk is 7.3 TiB (8001563222016 bytes). DOS partition table format cannot be used on drives for volumes larger than 2199023255040 bytes for 512-byte sectors. Use GUID partition table format (GPT). => mkfs -t vfat -F 32 /dev/nvme0n1p1 ... => mkfs -t ext4 /dev/nvme0n1p2 ... Syncing file systems (can take a long time) Syncing mounted partitions: Mounting /dev/nvme0n1p2 on /mnt/clone => rsync // /mnt/clone with-root-excludes ... Mounting /dev/nvme0n1p1 on /mnt/clone/boot => rsync /boot/ /mnt/clone/boot ... Editing /mnt/clone/boot/cmdline.txt PARTUUID to use db6793cc Editing /mnt/clone/etc/fstab PARTUUID to use db6793cc =============================== Done with clone to /dev/nvme0n1 Start - 16:36:11 End - 16:36:54 Elapsed Time - 0:43 Cloned partitions are mounted on /mnt/clone for inspection or customizing. Hit Enter when ready to unmount the /dev/nvme0n1 partitions ... unmounting /mnt/clone/boot unmounting /mnt/clone =============================== pi@dvr:~/rpi-clone $ lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT mmcblk0 179:0 0 14.6G 0 disk ├─mmcblk0p1 179:1 0 256M 0 part /boot └─mmcblk0p2 179:2 0 14.3G 0 part / mmcblk0boot0 179:32 0 4M 1 disk mmcblk0boot1 179:64 0 4M 1 disk nvme0n1 259:0 0 7.3T 0 disk ├─nvme0n1p1 259:1 0 256M 0 part └─nvme0n1p2 259:2 0 2T 0 part ``` Note that I was testing with an 8 TB NVMe drive, and it was only sized to 2 TB...
framps commented 2022-10-12 20:36:52 +02:00 (Migrated from github.com)

I just tested the PR on my CM4 with the 04-04 lite image. Boot from the cloned nvme works perfect 👍

Unfortunately my CM4 doesn't boot any more from nvme with later images. I created an issue on this.

I just tested the PR on my CM4 with the 04-04 lite image. Boot from the cloned nvme works perfect :+1: Unfortunately my CM4 doesn't boot any more from nvme with later images. I created an [issue](https://github.com/RPi-Distro/repo/issues/309) on this.
notenoughtech commented 2022-10-27 19:01:43 +02:00 (Migrated from github.com)

I'm trying this on CM4 with eMMC storage and /dev/nvme0n1 and it fails despite the manual changes made to the rpi-clone file. I changed the 576 and 1060 line with the same OR condition, but without much luck. Pointers appreciated

pi@ant1:~ $ lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT mmcblk0 179:0 0 29.1G 0 disk ├─mmcblk0p1 179:1 0 256M 0 part /boot └─mmcblk0p2 179:2 0 28.9G 0 part / mmcblk0boot0 179:32 0 4M 1 disk mmcblk0boot1 179:64 0 4M 1 disk nvme0n1 259:0 0 232.9G 0 disk ├─nvme0n1p1 259:1 0 256M 0 part └─nvme0n1p2 259:2 0 232.6G 0 part

Syncing file systems (can take a long time) Syncing mounted partitions: Mounting /dev/nvme0n12 on /mnt/clone mount: /mnt/clone: special device /dev/nvme0n12 does not exist. Mount failure of /dev/nvme0n12 on /mnt/clone. Aborting!

I'm trying this on CM4 with eMMC storage and /dev/nvme0n1 and it fails despite the manual changes made to the rpi-clone file. I changed the 576 and 1060 line with the same OR condition, but without much luck. Pointers appreciated `pi@ant1:~ $ lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT mmcblk0 179:0 0 29.1G 0 disk ├─mmcblk0p1 179:1 0 256M 0 part /boot └─mmcblk0p2 179:2 0 28.9G 0 part / mmcblk0boot0 179:32 0 4M 1 disk mmcblk0boot1 179:64 0 4M 1 disk nvme0n1 259:0 0 232.9G 0 disk ├─nvme0n1p1 259:1 0 256M 0 part └─nvme0n1p2 259:2 0 232.6G 0 part ` `Syncing file systems (can take a long time) Syncing mounted partitions: Mounting /dev/nvme0n12 on /mnt/clone mount: /mnt/clone: special device /dev/nvme0n12 does not exist. Mount failure of /dev/nvme0n12 on /mnt/clone. Aborting! `
framps commented 2022-10-27 20:31:54 +02:00 (Migrated from github.com)

I changed the 576 and 1060 line with the same OR condition,

Why didn't you grab the file from @geerlingguy s PR instead of updating the file? I suggest to compare your modified rpi-clone version with the one in the PR or grab the file from the PR and test again. Just to make sure you use the PR code 😉 .

I have exactly the same config other than a smaller NVMe disk and it works for me.

pi@raspberrypi:~ $ lsblk
NAME         MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
mmcblk0      179:0    0  29.1G  0 disk 
|-mmcblk0p1  179:1    0   256M  0 part 
`-mmcblk0p2  179:2    0  28.9G  0 part 
mmcblk0boot0 179:32   0     4M  1 disk 
mmcblk0boot1 179:64   0     4M  1 disk 
nvme0n1      259:0    0 119.2G  0 disk 
|-nvme0n1p1  259:1    0   256M  0 part /boot
`-nvme0n1p2  259:2    0   119G  0 part /
> I changed the 576 and 1060 line with the same OR condition, Why didn't you grab [the file](https://github.com/geerlingguy/rpi-clone/blob/123-nvme/rpi-clone) from @geerlingguy s PR instead of updating the file? I suggest to compare your modified rpi-clone version with the one in the PR or grab the file from the PR and test again. Just to make sure you use the PR code :wink: . I have exactly the same config other than a smaller NVMe disk and it works for me. ``` pi@raspberrypi:~ $ lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT mmcblk0 179:0 0 29.1G 0 disk |-mmcblk0p1 179:1 0 256M 0 part `-mmcblk0p2 179:2 0 28.9G 0 part mmcblk0boot0 179:32 0 4M 1 disk mmcblk0boot1 179:64 0 4M 1 disk nvme0n1 259:0 0 119.2G 0 disk |-nvme0n1p1 259:1 0 256M 0 part /boot `-nvme0n1p2 259:2 0 119G 0 part / ```
notenoughtech commented 2022-10-28 18:46:35 +02:00 (Migrated from github.com)

Why didn't you grab the file from @geerlingguy s PR instead of updating the file? I suggest to compare your modified rpi-clone version with the one in the PR or grab the file from the PR and test again. Just to make sure you use the PR code 😉 .

Both methods the file from PR and manually editing the script produce the same error.

> Why didn't you grab [the file](https://github.com/geerlingguy/rpi-clone/blob/123-nvme/rpi-clone) from @geerlingguy s PR instead of updating the file? I suggest to compare your modified rpi-clone version with the one in the PR or grab the file from the PR and test again. Just to make sure you use the PR code 😉 . Both methods the file from PR and manually editing the script produce the same error.
framps commented 2022-10-28 20:06:00 +02:00 (Migrated from github.com)

😢 Well, then a bash debug log may help. Just invoke the script with sudo bash -x rpi-clone <your args> and attach the debug log.

:cry: Well, then a bash debug log may help. Just invoke the script with `sudo bash -x rpi-clone <your args>` and attach the debug log.
notenoughtech commented 2022-10-28 20:22:58 +02:00 (Migrated from github.com)

Plot thickens:
CM4 Lite 4GB RAM + WiFi = clone successful
CM4 4GB RAM + eMMC + WiFi = clone fails. I'll get the logs in a sec

Plot thickens: CM4 Lite 4GB RAM + WiFi = clone successful CM4 4GB RAM + eMMC + WiFi = clone fails. I'll get the logs in a sec
framps commented 2022-10-28 22:14:26 +02:00 (Migrated from github.com)

attach the debug log

I suggest to invoke the script command first, then lsblk then invoke sudo bash -x rpi-clone nvme0n1 and finally exit. You now have created a file transcript which is the debug log I'm asking for.

> attach the debug log I suggest to invoke the `script` command first, then `lsblk` then invoke `sudo bash -x rpi-clone nvme0n1` and finally `exit`. You now have created a file `transcript` which is the debug log I'm asking for.
notenoughtech commented 2022-10-28 23:09:01 +02:00 (Migrated from github.com)

Right, I'm even more confused.
sudo bash -x rpi-clone -l nvme0n1 works on both types of CM4 modules, while
sudo rpi-clone -l nvme0n1 did not work. Getting late so I'll collect the logs tomorrow and share my findings.

Right, I'm even more confused. `sudo bash -x rpi-clone -l nvme0n1` works on both types of CM4 modules, while `sudo rpi-clone -l nvme0n1` did not work. Getting late so I'll collect the logs tomorrow and share my findings.
framps commented 2022-11-02 23:15:16 +01:00 (Migrated from github.com)

Right, I'm even more confused.

@notenoughtech Any updates on your issue?

> Right, I'm even more confused. @notenoughtech Any updates on your issue?
notenoughtech commented 2022-11-03 20:08:29 +01:00 (Migrated from github.com)

Hi, @framps
I'm sorry for the late response. Here is the log as requested:

Hi, @framps I'm sorry for the late response. Here is the log as requested:
notenoughtech commented 2022-11-03 20:13:55 +01:00 (Migrated from github.com)

Right.
Even more confused now as this all worked well and I finally have the boot from NVMe.
log.txt

Right. Even more confused now as this all worked well and I finally have the boot from NVMe. [log.txt](https://github.com/billw2/rpi-clone/files/9932391/log.txt)
framps commented 2022-11-03 20:21:39 +01:00 (Migrated from github.com)

Oh good 😳 ... you can attach a file in a comment - just use drag and drop 😉

Please clean up your huge comment ... nobody will ever read this 😢

Oh good :flushed: ... you can attach a file in a comment - just use drag and drop :wink: Please clean up your huge comment ... nobody will ever read this :cry:
framps commented 2022-11-03 20:22:14 +01:00 (Migrated from github.com)

all worked well

So there is no need any more for the debug log 😄

> all worked well So there is no need any more for the debug log :smile:
notenoughtech commented 2022-11-03 20:27:42 +01:00 (Migrated from github.com)

I thought the code would auto collapse or something. I cleaned it up. Now let's see if this works with non eMMC CM4s

I thought the code would auto collapse or something. I cleaned it up. Now let's see if this works with non eMMC CM4s
notenoughtech commented 2022-11-04 18:58:01 +01:00 (Migrated from github.com)

No matter how many times I try, the CM4 Lite is not liking the boot from nvme. rpi-clone worked on CM4 and change of boot from eMMC to NVMe was easy enough. Same EEPROM firmware on CM4 Lite and boot sequence on CM4 Lite and no dice after rpi-clone.
While cloning appears to be working, I don't think the boot partition is able to boot. Without SD card it hangs but hot swapping sd card, proves that boot sequence is working as the CM4 Lite boots from card a moment later without a reboot.

Anyone had this much problems?

I can also confirm that I still have the issue of sudo rpi-clone -l nvme0n1 not working while sudo bash -x rpi-clone nvme0n1 works fine

No matter how many times I try, the CM4 Lite is not liking the boot from nvme. rpi-clone worked on CM4 and change of boot from eMMC to NVMe was easy enough. Same EEPROM firmware on CM4 Lite and boot sequence on CM4 Lite and no dice after rpi-clone. While cloning appears to be working, I don't think the boot partition is able to boot. Without SD card it hangs but hot swapping sd card, proves that boot sequence is working as the CM4 Lite boots from card a moment later without a reboot. Anyone had this much problems? I can also confirm that I still have the issue of `sudo rpi-clone -l nvme0n1` not working while `sudo bash -x rpi-clone nvme0n1` works fine
framps commented 2022-11-04 19:48:30 +01:00 (Migrated from github.com)

I don't have a CM4 lite to test rpi-clone in this environment.

Not sure why you actually use rpi-clone. If you just clone the SD card image to NVMe to be able to boot from NVMe why don't you install the OS directly on NVMe instead to the SD card first and then clone the SD to NVMe? See this page how I install RaspbianOS directly on NVMe.

I don't have a CM4 lite to test rpi-clone in this environment. Not sure why you actually use rpi-clone. If you just clone the SD card image to NVMe to be able to boot from NVMe why don't you install the OS directly on NVMe instead to the SD card first and then clone the SD to NVMe? See [this page](https://www.linux-tips-and-tricks.de/en/raspberrye/614-raspberry-compute-module-4-setup-guide) how I install RaspbianOS directly on NVMe.
notenoughtech commented 2022-11-04 19:56:53 +01:00 (Migrated from github.com)

Why we use tools?
We both know working with rpi-clone is dead easy instead of messing about
with dd.
That's for why.

I could just flash the NVMe drives via CM4 as this worked, but the idea is
to get it working properly rather than hack it working.
What I don't understand why executing rpi-clone via bash works on CM4 but
failed when same file runs from the command line.
Mat Zolnierczyk

Maker, Robotics Engineer

Owner | NotEnoughTech
@.***
notenoughtech.com
Teesside, North Yorkshire, UK
[image: facebook] https://www.facebook.com/NotEnoughTECH/
[image: twitter] https://twitter.com/NotEnoughTECH
[image: linkedin] https://www.linkedin.com/in/mat-zolnierczyk/
[image: instagram] https://www.instagram.com/notenoughtech/

On Fri, Nov 4, 2022 at 6:48 PM framp @.***> wrote:

I don't have a CM4 lite to test rpi-clone in this environment.

Not sure why you actually use rpi-clone. If you just clone the SD card
image to NVMe to be able to boot from NVMe why don't you install the OS
directly on NVMe instead to the SD card first and then clone the SD to
NVMe? See this page
https://www.linux-tips-and-tricks.de/en/raspberrye/614-raspberry-compute-module-4-setup-guide
how I install RaspbianOS directly on NVMe.


Reply to this email directly, view it on GitHub
https://github.com/billw2/rpi-clone/pull/147#issuecomment-1304001725,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AKDRL244BUVHF53KMYJ7HZDWGVLATANCNFSM6AAAAAAQSQJWOM
.
You are receiving this because you were mentioned.Message ID:
@.***>

Why we use tools? We both know working with rpi-clone is dead easy instead of messing about with dd. That's for why. I could just flash the NVMe drives via CM4 as this worked, but the idea is to get it working properly rather than hack it working. What I don't understand why executing rpi-clone via bash works on CM4 but failed when same file runs from the command line. Mat Zolnierczyk Maker, Robotics Engineer Owner | NotEnoughTech ***@***.*** notenoughtech.com Teesside, North Yorkshire, UK [image: facebook] <https://www.facebook.com/NotEnoughTECH/> [image: twitter] <https://twitter.com/NotEnoughTECH> [image: linkedin] <https://www.linkedin.com/in/mat-zolnierczyk/> [image: instagram] <https://www.instagram.com/notenoughtech/> On Fri, Nov 4, 2022 at 6:48 PM framp ***@***.***> wrote: > I don't have a CM4 lite to test rpi-clone in this environment. > > Not sure why you actually use rpi-clone. If you just clone the SD card > image to NVMe to be able to boot from NVMe why don't you install the OS > directly on NVMe instead to the SD card first and then clone the SD to > NVMe? See this page > <https://www.linux-tips-and-tricks.de/en/raspberrye/614-raspberry-compute-module-4-setup-guide> > how I install RaspbianOS directly on NVMe. > > — > Reply to this email directly, view it on GitHub > <https://github.com/billw2/rpi-clone/pull/147#issuecomment-1304001725>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AKDRL244BUVHF53KMYJ7HZDWGVLATANCNFSM6AAAAAAQSQJWOM> > . > You are receiving this because you were mentioned.Message ID: > ***@***.***> >
framps commented 2022-11-04 20:11:42 +01:00 (Migrated from github.com)

What I don't understand why executing rpi-clone via bash works on CM4 but failed when same file runs from the command line.

This requires further debugging I'm not able to assist. @billw2 may be able to help. But given his last commit is 2.5 years ago I doubt he will.

> What I don't understand why executing rpi-clone via bash works on CM4 but failed when same file runs from the command line. This requires further debugging I'm not able to assist. @billw2 may be able to help. But given his last commit is 2.5 years ago I doubt he will.
geerlingguy commented 2024-02-07 18:17:12 +01:00 (Migrated from github.com)

This patch is now maintained in my fork: https://github.com/geerlingguy/rpi-clone

This patch is now maintained in my fork: https://github.com/geerlingguy/rpi-clone
This pull request can be merged automatically.
You are not authorized to merge this pull request.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin geerlingguy/123-nvme:geerlingguy/123-nvme
git checkout geerlingguy/123-nvme
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Lerking/rpi-clone#147
No description provided.