Skip to content

media: i2c: st-vd55g: Genericize driver and add VD55G0 support - #169

Open
petm5 wants to merge 10 commits into
linux-surface:v6.18-surface-develfrom
petm5:v6.18-surface-devel-vd55g0
Open

media: i2c: st-vd55g: Genericize driver and add VD55G0 support#169
petm5 wants to merge 10 commits into
linux-surface:v6.18-surface-develfrom
petm5:v6.18-surface-devel-vd55g0

Conversation

@petm5

@petm5 petm5 commented Aug 2, 2026

Copy link
Copy Markdown

This set of changes refactors the STMicroelectronics VD55G1 and VD65G4 image sensor driver (vd55g1) into a generic VD55G family driver (vd55g) and adds support for the VD55G0 monochrome sensor.

The VD55G0 sensor is found in devices such as the Microsoft Surface Pro 9 under ACPI HID SMO55F0. It features a native resolution of 644 x 604 pixels and differs from later variants in the family in several ways:

  • It uses a different native resolution and register addresses mapping,
    requiring specification of generational feature differences.
  • It lacks support for spatial HDR and internal subtraction.
  • It requires an exposure duty cycle limit when strobe illumination is enabled
    to limit illuminator current and avoid tripping overcurrent protection.

Key changes:

  1. DT Bindings: Rename binding schema to represent the ST VD55G family and document the st,vd55g0 compatibility string.
  2. IPU bridge: Add SMO55F0 ACPI HID to the list of supported sensors on IPU6 platforms.
  3. INT3472 platform driver: Handle supply naming conventions on ACPI systems with a GPIO mapping quirk.
  4. Media subsystem: Refactor vd55g1 into generic vd55g, introduce per-chip abstractions, load firmware externally, and implement VD55G0 support.
  5. Cleanup: Remove superseded vd55g1 code remnants.
  6. Maintainers: Update paths for the generic ST VD55G family driver/bindings.

Tested on a Microsoft Surface Pro 9 (an Intel IPU6 platform with ACPI HID SMO55F0) running libcamera. Validated binding schemas using make dt_binding_check.

bemug-st and others added 10 commits August 2, 2026 16:58
Switch compatible from a const to an enum to accommodate both the vd55g1
and the vd65g4, which is the color variant.

Signed-off-by: Benjamin Mugnier <benjamin.mugnier@foss.st.com>
Acked-by: Conor Dooley <conor.dooley@microchip.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Hans Verkuil <hverkuil+cisco@kernel.org>
The vd65g4 is the bayer version of the vd55g1.
As opposed to the vd55g1, the vd65g4 does not need any patch. Check the
sensor id at probe and choose to patch or not on power_on() according to
it.
It's bayer matrix's order is RGGB. This commit handles hflip and vflip
by switching the bayer pattern accordingly.

Signed-off-by: Benjamin Mugnier <benjamin.mugnier@foss.st.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Hans Verkuil <hverkuil+cisco@kernel.org>
A firmware patch consisting of an array of bytes is uploaded to the chip
during its boot sequence. Declutter the driver source by moving it to a
separate header file.
Document the STMicroelectronics VD55G0 camera sensor. Rename the device
tree bindings to represent a generic VD55G family sensor.

Signed-off-by: Peter Marshall <pm@petermarshall.ca>
Add SMO55F0 ACPI HID for the ST VD55G0 sensor to the list of supported
sensors. The sensor is found with this HID on a Microsoft Surface Pro
9 tablet. Use the lowest supported link frequency.

Signed-off-by: Peter Marshall <pm@petermarshall.ca>
The vd55g sensor drivers / DT bindings expect non-standard supply names.
Add a mapping to enable them.

Signed-off-by: Peter Marshall <pm@petermarshall.ca>
Rewrite the existing vd55g1 module supporting ST VD55G1 and VD65G4 to
make it generic for ST VD55G family global-shutter camera sensors.

Registers and supported features differ between sensor generations.
Replace the static register addresses and revision-specific branches
with a per-chip info struct.

Add support for the VD55G0 monochrome camera sensor. The VD55G0 has
a native resolution of 644 x 604 pixels and does not support HDR via
internal subtraction.

Add a quirk limiting exposure duty cycle with strobe illumination
enabled to avoid tripping overcurrent protection.

Co-authored-by: Fernando Rimoli <119756054+femito1@users.noreply.github.com>
Signed-off-by: Peter Marshall <pm@petermarshall.ca>
vd55g1 is superseded by the generic vd55g driver supporting the same
hardware and feature set. Remove it.

Signed-off-by: Peter Marshall <pm@petermarshall.ca>
Point to the updated vd55g sources and documentation, and add myself as
a maintainer.

Signed-off-by: Peter Marshall <pm@petermarshall.ca>
The system's ring bus / RAM is clocked down during low power states,
causing flickering and FIFO timeut errors with some sensors when the CPU
is idle. This was originally noticed by Hans de Goede on the linux-media
mailing list (see https://marc.info/?l=linux-media&m=175999034702772).
Tighten the PM QoS value during IPU6 streaming to work around the issue.
A value of 50ms appears to work fine.

The IPU6 fails to start streaming with modes using 384 bytes per line or
less. Bump the minimum frame width up to a safe pixel limit.
@petm5 petm5 changed the title media: i2c: media: i2c: st-vd55g: Genericize driver and add VD55G0 support Aug 2, 2026
@fildunsky

Copy link
Copy Markdown

Confirming this works on a Surface Pro 8 (not Pro 9), and adding one
finding that cost me a lot of time and is not mentioned anywhere I could
find.

The driver from this PR binds cleanly to SMO55F0 on Pro 8, applies patch
2.11 from vd55g0-cut1.bin, and the subdev reports 644x604 Y10 as
expected. But the capture node delivered nothing: exactly one FRAME_SOF
about 255 ms after stream on, then silence — no FRAME_EOF, no
PIN_DATA_READY, and notably no csi2 errors at all, which makes it
look like the sensor never starts.

The sensor was fine the whole time. The pixel format on the ISYS capture
node has to be unpacked:

capture format mipi_store_mode result
Y10P 1 0 bytes, SOF=1
GREY (with subdev at Y8_1X8) 1 0 bytes, SOF=1
Y10 0 works, 50 fps, no errors

The mechanism is ipu6-isys-video.c:

input_pin->mipi_store_mode = pfmt->bpp == pfmt->bpp_packed ?
        IPU6_FW_ISYS_MIPI_STORE_MODE_DISCARD_LONG_HEADER :
        IPU6_FW_ISYS_MIPI_STORE_MODE_NORMAL;

Y10P and GREY both have bpp == bpp_packed and so select
DISCARD_LONG_HEADER, which does not work on this port. Y10 has
bpp 16 vs bpp_packed 10 and selects NORMAL.

Working invocation, for anyone else stuck at the same place:

media-ctl -d /dev/media0 -V '"vd55g 3-0060":0 [fmt:Y10_1X10/644x604]'
media-ctl -d /dev/media0 -V '"Intel IPU6 CSI2 5":0 [fmt:Y10_1X10/644x604]'
media-ctl -d /dev/media0 -V '"Intel IPU6 CSI2 5":1 [fmt:Y10_1X10/644x604]'
v4l2-ctl -d /dev/video42 --set-fmt-video=width=644,height=604,pixelformat='Y10 ' \
         --stream-mmap --stream-count=120 --stream-to=/tmp/ir.raw

Two things I chased first, in case it saves someone the detour. Neither
was the cause:

  • The context sequencer. vd55g_update_hdr_mode() only runs under
    if (info->hdr), so on VD55G0 the sensor keeps NEXT_CTX = 1 with
    REPEAT_COUNT_CTX0 = 0 after the firmware patch. That looks exactly
    like "emits one frame and stops", but setting the Windows values
    (1/1/1) changes nothing.
  • Link frequency. I tried both 380 MHz and 420 MHz. For what it is worth,
    the Windows driver writes 0x0224 = 840000000, i.e. 420 MHz given that
    the driver computes this register as link_freq * 2, and at 420 MHz the
    driver's line-length formula lands on 1200 against the Windows table's
    1201. But with the packed format neither frequency works, so this was
    not diagnosable until the format was fixed.

What finally located it was reading sensor registers during streaming: a
statistics register at 0x00b4 was changing ~50 times a second, proving the
sensor was capturing continuously and only transmission was broken. Then
diffing the IPU6_FW_ISYS_STREAM_CFG_DATA block against a working colour
camera on the same machine pointed straight at mipi_store_mode.

I also wrote the entire register table from the Windows driver
(vd55g0.sys, .rdata at file offset 0x47434, 71 entries) into the
sensor verbatim just before stream start, to rule out sensor
configuration completely — behaviour did not change.

Notes and tooling: https://github.com/fildunsky/linux-surface-pro8-cameras

Thank you for the driver — the rest of it worked first time.

@femito1

femito1 commented Sep 1, 2026

Copy link
Copy Markdown

@petm5 Two things: a question about your plans for the list, and a follow-up on the
small-mode limit that builds on what @fildunsky worked out in this thread.

Are you planning to post the vd55g series to linux-media, and roughly when?

I have two fixes parked behind yours and do not want to jump the queue. Neither is in your
driver:

  • int3472: a layered consumer lookup in common.c (dep_list, then a _DEP walk, then
    -EPROBE_DEFER, then ENODEV). Without it the IR sensor gets a deviceless clock and
    fails -121 on every boot here.
  • ov13858: a chip-id read retry, 5 attempts 5ms apart. Without it the rear camera fails
    -5 on every boot.

I reverted each on its own; both are independently necessary here. I have not sent them
because neither failure is reproducible on mainline until IR enablement is present, so
today they would be fixes for bugs no maintainer can trigger. Once your series is on the
list I can send them behind it and reference it.

Your own platform/x86: int3472: Add VD55G0 supply GPIO mapping is about supply naming, so
there is no overlap. If you would rather carry either yourself, or want me to hold off,
that is fine. I just do not want to sit on them indefinitely.

IPU6_ISYS_MIN_WIDTH 2U -> 392U: the mechanism is @fildunsky's, and the number does not travel

@fildunsky already identified the mechanism in this thread on 31 Aug: packed capture formats
(bpp == bpp_packed) select IPU6_FW_ISYS_MIPI_STORE_MODE_DISCARD_LONG_HEADER and pin type
MIPI, unpacked ones select NORMAL and RAW_SOC, and on his Pro 8 only the unpacked path
delivers frames. I ran the bytes-versus-pixels test I offered you in July without having
seen his comment, got the same discriminator from the other end, and then found that his
result and mine disagree in a way that matters for this patch. So this is a follow-up to his
finding, not a competing one.

I know the 392 is deliberately temporary, so take the rest as notes toward the real fix.

On this Surface Pro 9, 392 is exactly right for 8-bit. ipu6_isys_try_fmt_cap() does
ALIGN(*bytesperline, 64), so at 8bpp widths 321-384 all give 384 and 385-448 all give 448.
The boundary is a bucket edge, and 392 is the first width reaching 448. Measured on the IR
camera with GREY: 352 and 384 fail, 392 streams 3/3, 448 streams 2/2.

It is not a width limit. Holding width fixed and changing only the memory format flips
the result:

width 640 384 320 192 96 64
GREY (packed) OK fail fail fail not built not built
Y10P (packed) OK OK OK fail not built not built
Y10 (unpacked) OK OK OK OK OK OK

Y10 streams at 64 px. Nor is it bytes per line as such: aligned bytesperline 384 fails as
GREY at 384 px and works as Y10 at 192 px, and the same holds at aligned 192, 256 and
320. Height is irrelevant too: 384 wide fails at heights 288, 300 and 480 while 400x288
works. (That last point also corrects my July numbers, where width and height co-varied.)

But the threshold is not portable which is the part I think changes your fix.
@fildunsky reports packed failing at the native 644x604, where aligned bytesperline is
704 for GREY and 832 for Y10P. Both are far above the 448 boundary I measured, so I ran
his exact configuration here: same mode, same manual media-ctl pipeline, same v4l2-ctl
capture on the raw ISYS node.

subdev capture aligned bpl his Pro 8 this Pro 9
Y10_1X10/644x604 Y10 1344 works, 50 fps streamed, 8117760 B
Y10_1X10/644x604 Y10P 832 0 bytes, SOF=1 streamed, 5025280 B
Y8_1X8/644x604 GREY 704 0 bytes, SOF=1 streamed, 4252160 B

Every byte count is exactly bytesperline * height * frames, so those are full frames. I
also ran GREY at 640x480 as a method control, since that is what my relay uses daily, to
make sure the raw-node path was not the thing failing. It passed.

So his two failing configurations stream here at the same aligned byte counts. Either his
threshold is much higher than mine or his packed path does not work at all. There is no
single number that is correct for both machines
, which means a flat constant in a
clamp() cannot be right for both of us regardless of which value it holds.

What differs, none of it isolated: he is on IPU6 TGL 8086:9a19 and I am on IPU6EP ADL-P
8086:465d; he is on CSI-2 port 5 and I am on port 3, and this ADL-P only exposes ports 0-3
at all, so his port does not exist on my SoC; his link frequency is 380 or 420 MHz against my
600 MHz. I cannot test the link frequency cheaply, since it comes from the software node the
ipu-bridge builds, V4L2_CID_LINK_FREQ is read-only and there is no module parameter. For
what it is worth, a minimum-line-duration model would predict his lower link rate to be
safer rather than more fragile, so I do not think that is it, but that is an argument and not
a measurement.

Where that leaves the guard. The predicate is the reliable part, and it is already
available in try_fmt_cap(): gate on pfmt->bpp == pfmt->bpp_packed, so the SoC path stops
being restricted at all. As written the flat pixel floor silently widens a perfectly good
unpacked Y10 request for 320x240 into 392x240, because MIN_WIDTH is a clamp() rather
than a reject, and ENUM_FRAMESIZES then advertises 392 as min_width for every format.
Whether the packed side can carry a number at all is the open question: 448 holds here and
would not have helped @fildunsky.

Two more things:

  • The predicate predicts who else is affected: every bpp == bpp_packed format takes the
    MIPI path, so GREY, all *8 Bayer, all *10P/*12P, plus UYVY, YUYV, RGB565 and
    Y16. Everything with a 16-bit container takes the SoC path. Presumably why the
    visible-light cameras never showed this, since libcamera uses unpacked SBGGR10 there. I
    measured only GREY, Y10P and Y10; the rest is inference from the pfmt table.
  • The 448 is not in the driver. No ipu6 source has a constant for a minimum line length
    or byte count, and the same stride goes down both paths, so it is enforced in the
    firmware or the MIPI path hardware. That fits the failure being
    wait_for_completion_timeout() on a stream_stop/close ack that never arrives, and it fits
    the boundary being platform-dependent. I ruled out the iwake watermark, which uses
    DIV_ROUND_UP(bytes_per_line, 2048) and floors at one page, and
    IPU6_ISYS_OVERALLOC_MIN, which only inflates sizeimage.

One method warning, since it cost me an hour. media-ctl -V silently snaps to the nearest
advertised mode, the same way libcamera does without --strict-formats. Asking a driver that
advertises 644x604, 640x600 and 640x480 for 392x604 leaves the subdev at 640x600 while the
capture node accepts 392x604, and the mismatched pipeline delivers zero bytes, which looks
exactly like a real failure. Reading the subdev format back after setting it is the only way
to tell those apart.

Happy to test a candidate patch on this SP9 IR camera, or to check the UYVY/Y16
prediction, or to build extra modes at whatever geometry would be useful.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants