1
00:00:03,767 --> 00:00:05,718
This is testbed-qemu.

2
00:00:05,718 --> 00:00:09,619
It boots a real prplOS router inside QEMU.

3
00:00:10,120 --> 00:00:14,300
We call that router the
device under test, or DUT.

4
00:00:14,800 --> 00:00:19,373
It runs the prplOS cram tests,
the same tests that CI runs.

5
00:00:19,873 --> 00:00:24,837
It boots the DUT on your computer,
from the prplOS image you choose.

6
00:00:29,197 --> 00:00:33,692
Docker Compose or rootless Podman
runs the testbed-qemu container.

7
00:00:33,692 --> 00:00:38,740
Inside the container, the test host
runs cram, SSH and the serial console.

8
00:00:38,740 --> 00:00:43,235
Next to it, QEMU boots the DUT,
and uses KVM when the host has it.

9
00:00:43,235 --> 00:00:45,586
Three wires connect the two sides.

10
00:00:45,586 --> 00:00:47,937
The LAN carries your test traffic.

11
00:00:47,937 --> 00:00:52,086
The WAN gets its address from
a DHCP server in the container,

12
00:00:52,086 --> 00:00:54,922
and reaches the outside world through it.

13
00:00:54,922 --> 00:00:57,480
The third wire is the serial console.

14
00:00:57,480 --> 00:01:02,805
The host gives the container the prplOS
tree and, read-only, the prplOS image.

15
00:01:02,805 --> 00:01:07,092
It also gives a folder for logs
and results, and for the shell,

16
00:01:07,092 --> 00:01:09,374
a folder for its persistent disk.

17
00:01:13,734 --> 00:01:19,603
You need a Linux host with Docker Compose
or rootless Podman, plus git and make.

18
00:01:19,603 --> 00:01:23,020
With KVM, the DUT boots
in about half a minute.

19
00:01:23,020 --> 00:01:26,065
Without it, QEMU still works, but slowly.

20
00:01:26,565 --> 00:01:29,388
Clone the testbed-qemu repository.

21
00:01:32,900 --> 00:01:34,351
Then run make fetch.

22
00:01:34,351 --> 00:01:38,849
It downloads the pinned prplOS
image and verifies its checksum.

23
00:01:38,849 --> 00:01:43,637
It also writes a trimmed prplOS
test tree into the qa-cache folder.

24
00:01:44,137 --> 00:01:46,805
Then run make test-qemu-shell.

25
00:01:46,805 --> 00:01:50,183
On first use, it builds the container.

26
00:01:50,183 --> 00:01:54,718
It boots the DUT and opens
a shell in the container.

27
00:01:59,078 --> 00:02:02,961
Test buckets such as smoke
need a real prplOS clone,

28
00:02:02,961 --> 00:02:05,778
not the trimmed tree from make fetch.

29
00:02:05,778 --> 00:02:09,737
So clone prplOS at the commit
that testbed-qemu pins.

30
00:02:10,237 --> 00:02:11,887
Then run make test-qemu.

31
00:02:11,887 --> 00:02:15,324
Point it at the clone,
and select the smoke bucket.

32
00:02:15,324 --> 00:02:19,173
It boots a fresh DUT and waits
for SSH and the datamodel.

33
00:02:19,173 --> 00:02:23,848
Then it runs the six smoke tests
that the clone lists for this board.

34
00:02:26,625 --> 00:02:31,534
In prplOS CI, a merge request can
start the same smoke bucket by hand,

35
00:02:31,534 --> 00:02:35,090
as a QEMU job, in this same
testbed-qemu container.

36
00:02:35,090 --> 00:02:38,576
It runs the six smoke tests
and skips one of them.

37
00:02:38,576 --> 00:02:40,568
Then it runs two post tests.

38
00:02:40,568 --> 00:02:45,831
The run passed, and the job still
collected debug information from the DUT.

39
00:02:45,831 --> 00:02:48,961
The job runs on a GitLab
runner that has KVM.

40
00:02:53,321 --> 00:02:55,851
Shell mode is make test-qemu-shell.

41
00:02:55,851 --> 00:02:59,827
You get a shell at once,
while the DUT is still booting.

42
00:03:01,090 --> 00:03:02,779
Press the up arrow.

43
00:03:02,779 --> 00:03:08,998
The shell history already holds
testbed-qemu wait and testbed-qemu ssh.

44
00:03:08,998 --> 00:03:13,796
The wait command blocks until
the DUT answers over SSH.

45
00:03:13,796 --> 00:03:17,261
Then ssh opens a root shell on the DUT.

46
00:03:17,761 --> 00:03:22,569
Now you work on the DUT as root,
for example to list its routes.

47
00:03:26,761 --> 00:03:32,878
If you need the serial console instead
of SSH, run testbed-qemu console.

48
00:03:32,878 --> 00:03:35,032
Press Enter for a prompt.

49
00:03:35,032 --> 00:03:38,047
Ctrl-] takes you back to the shell.

50
00:03:42,407 --> 00:03:43,534
Now add a test.

51
00:03:43,534 --> 00:03:49,090
Save it in the prplOS tree on your
computer: the shell sees the same files.

52
00:03:49,090 --> 00:03:53,971
A cram test is a text file:
a command, then the output you expect.

53
00:03:53,971 --> 00:03:57,951
This one checks that SSH
reaches the DUT over the LAN.

54
00:03:58,451 --> 00:04:00,598
Run it with testbed-qemu cram.

55
00:04:00,598 --> 00:04:03,603
It runs against the
DUT that is already up,

56
00:04:03,603 --> 00:04:06,394
so you can rerun it without a new boot.

57
00:04:06,894 --> 00:04:10,578
Now break it: change the
expected output in the file.

58
00:04:13,394 --> 00:04:14,317
Run it again.

59
00:04:14,317 --> 00:04:19,716
Cram prints a diff: the line that the
test expects, and the line that it got.

60
00:04:21,894 --> 00:04:27,187
If the new output is right, accept it:
run the cp command that the runner prints,

61
00:04:27,187 --> 00:04:27,981
on the host.

62
00:04:33,754 --> 00:04:35,998
When a QEMU job fails in CI,

63
00:04:35,998 --> 00:04:41,931
reproduce it on your computer first:
run the same test with make test-qemu.

64
00:04:42,431 --> 00:04:46,508
Here we run the reboot test
from testbed-qemu's own tests.

65
00:04:46,508 --> 00:04:51,658
It writes a marker file, reboots the
DUT, and waits for SSH to come back.

66
00:04:51,658 --> 00:04:56,594
Then it checks that the marker
survived and that the uptime was reset.

67
00:04:56,594 --> 00:05:00,457
testbed-qemu's own CI runs
this test in every pipeline.

68
00:05:00,957 --> 00:05:04,633
A batch run with make test-qemu
always boots a private,

69
00:05:04,633 --> 00:05:06,811
factory-fresh copy of the image.

70
00:05:06,811 --> 00:05:10,759
The shell flashes its own disk
from the image on first use,

71
00:05:10,759 --> 00:05:15,320
and keeps that disk between sessions,
in the qemu-disk-state folder.

72
00:05:15,320 --> 00:05:18,248
To start clean, run
testbed-qemu disk reset.

73
00:05:18,248 --> 00:05:21,856
After that, the next controlled
boot is factory-fresh.

74
00:05:22,356 --> 00:05:23,568
When a run fails,

75
00:05:23,568 --> 00:05:27,204
testbed-qemu collects debug
information from the DUT

76
00:05:27,204 --> 00:05:29,343
before it powers the DUT down.

77
00:05:29,343 --> 00:05:35,332
The prplOS CI job you saw earlier collects
it on every run, even when the run passes.

78
00:05:35,332 --> 00:05:38,683
In a live shell, you can
ask for it at any time.

79
00:05:43,043 --> 00:05:45,891
testbed-qemu also ships
a skill for agents.

80
00:05:45,891 --> 00:05:50,298
The skill file lists the exact
commands, exit codes and a runbook.

81
00:05:50,298 --> 00:05:53,485
Here Claude Code, on Sonnet,
works in auto mode.

82
00:05:53,485 --> 00:05:57,147
It loads the skill and checks
the host with the doctor,

83
00:05:57,147 --> 00:05:59,791
which builds the container image first.

84
00:05:59,791 --> 00:06:04,402
Then the test boots the DUT in QEMU,
waits for SSH and the datamodel,

85
00:06:04,402 --> 00:06:06,504
and runs the connectivity test.

86
00:06:06,504 --> 00:06:10,097
It reports that the test
passed, with the post checks,

87
00:06:10,097 --> 00:06:12,335
the boot times and the artifacts.

88
00:06:12,335 --> 00:06:16,742
The status line shows what the run
cost in tokens, time and money.

89
00:06:19,403 --> 00:06:24,467
testbed-qemu boots the prplOS image
you fetch, with its full userspace.

90
00:06:24,467 --> 00:06:28,157
It gives the DUT a real LAN,
WAN and serial console.

91
00:06:28,157 --> 00:06:30,616
It does not replace real hardware,

92
00:06:30,616 --> 00:06:34,451
and it models neither a
switch nor a cellular gateway.