media: i2c: st-vd55g: Genericize driver and add VD55G0 support - #169
media: i2c: st-vd55g: Genericize driver and add VD55G0 support#169petm5 wants to merge 10 commits into
Conversation
Switch compatible from a const to an enum to accommodate both the vd55g1 and the vd65g4, which is the color variant. Signed-off-by: Benjamin Mugnier <benjamin.mugnier@foss.st.com> Acked-by: Conor Dooley <conor.dooley@microchip.com> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com> Signed-off-by: Hans Verkuil <hverkuil+cisco@kernel.org>
The vd65g4 is the bayer version of the vd55g1. As opposed to the vd55g1, the vd65g4 does not need any patch. Check the sensor id at probe and choose to patch or not on power_on() according to it. It's bayer matrix's order is RGGB. This commit handles hflip and vflip by switching the bayer pattern accordingly. Signed-off-by: Benjamin Mugnier <benjamin.mugnier@foss.st.com> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com> Signed-off-by: Hans Verkuil <hverkuil+cisco@kernel.org>
A firmware patch consisting of an array of bytes is uploaded to the chip during its boot sequence. Declutter the driver source by moving it to a separate header file.
Document the STMicroelectronics VD55G0 camera sensor. Rename the device tree bindings to represent a generic VD55G family sensor. Signed-off-by: Peter Marshall <pm@petermarshall.ca>
Add SMO55F0 ACPI HID for the ST VD55G0 sensor to the list of supported sensors. The sensor is found with this HID on a Microsoft Surface Pro 9 tablet. Use the lowest supported link frequency. Signed-off-by: Peter Marshall <pm@petermarshall.ca>
The vd55g sensor drivers / DT bindings expect non-standard supply names. Add a mapping to enable them. Signed-off-by: Peter Marshall <pm@petermarshall.ca>
Rewrite the existing vd55g1 module supporting ST VD55G1 and VD65G4 to make it generic for ST VD55G family global-shutter camera sensors. Registers and supported features differ between sensor generations. Replace the static register addresses and revision-specific branches with a per-chip info struct. Add support for the VD55G0 monochrome camera sensor. The VD55G0 has a native resolution of 644 x 604 pixels and does not support HDR via internal subtraction. Add a quirk limiting exposure duty cycle with strobe illumination enabled to avoid tripping overcurrent protection. Co-authored-by: Fernando Rimoli <119756054+femito1@users.noreply.github.com> Signed-off-by: Peter Marshall <pm@petermarshall.ca>
vd55g1 is superseded by the generic vd55g driver supporting the same hardware and feature set. Remove it. Signed-off-by: Peter Marshall <pm@petermarshall.ca>
Point to the updated vd55g sources and documentation, and add myself as a maintainer. Signed-off-by: Peter Marshall <pm@petermarshall.ca>
The system's ring bus / RAM is clocked down during low power states, causing flickering and FIFO timeut errors with some sensors when the CPU is idle. This was originally noticed by Hans de Goede on the linux-media mailing list (see https://marc.info/?l=linux-media&m=175999034702772). Tighten the PM QoS value during IPU6 streaming to work around the issue. A value of 50ms appears to work fine. The IPU6 fails to start streaming with modes using 384 bytes per line or less. Bump the minimum frame width up to a safe pixel limit.
|
Confirming this works on a Surface Pro 8 (not Pro 9), and adding one The driver from this PR binds cleanly to The sensor was fine the whole time. The pixel format on the ISYS capture
The mechanism is input_pin->mipi_store_mode = pfmt->bpp == pfmt->bpp_packed ?
IPU6_FW_ISYS_MIPI_STORE_MODE_DISCARD_LONG_HEADER :
IPU6_FW_ISYS_MIPI_STORE_MODE_NORMAL;
Working invocation, for anyone else stuck at the same place: media-ctl -d /dev/media0 -V '"vd55g 3-0060":0 [fmt:Y10_1X10/644x604]'
media-ctl -d /dev/media0 -V '"Intel IPU6 CSI2 5":0 [fmt:Y10_1X10/644x604]'
media-ctl -d /dev/media0 -V '"Intel IPU6 CSI2 5":1 [fmt:Y10_1X10/644x604]'
v4l2-ctl -d /dev/video42 --set-fmt-video=width=644,height=604,pixelformat='Y10 ' \
--stream-mmap --stream-count=120 --stream-to=/tmp/ir.rawTwo things I chased first, in case it saves someone the detour. Neither
What finally located it was reading sensor registers during streaming: a I also wrote the entire register table from the Windows driver Notes and tooling: https://github.com/fildunsky/linux-surface-pro8-cameras Thank you for the driver — the rest of it worked first time. |
|
@petm5 Two things: a question about your plans for the list, and a follow-up on the Are you planning to post the vd55g series to linux-media, and roughly when?I have two fixes parked behind yours and do not want to jump the queue. Neither is in your
I reverted each on its own; both are independently necessary here. I have not sent them Your own
|
| width | 640 | 384 | 320 | 192 | 96 | 64 |
|---|---|---|---|---|---|---|
GREY (packed) |
OK | fail | fail | fail | not built | not built |
Y10P (packed) |
OK | OK | OK | fail | not built | not built |
Y10 (unpacked) |
OK | OK | OK | OK | OK | OK |
Y10 streams at 64 px. Nor is it bytes per line as such: aligned bytesperline 384 fails as
GREY at 384 px and works as Y10 at 192 px, and the same holds at aligned 192, 256 and
320. Height is irrelevant too: 384 wide fails at heights 288, 300 and 480 while 400x288
works. (That last point also corrects my July numbers, where width and height co-varied.)
But the threshold is not portable which is the part I think changes your fix.
@fildunsky reports packed failing at the native 644x604, where aligned bytesperline is
704 for GREY and 832 for Y10P. Both are far above the 448 boundary I measured, so I ran
his exact configuration here: same mode, same manual media-ctl pipeline, same v4l2-ctl
capture on the raw ISYS node.
| subdev | capture | aligned bpl | his Pro 8 | this Pro 9 |
|---|---|---|---|---|
Y10_1X10/644x604 |
Y10 |
1344 | works, 50 fps | streamed, 8117760 B |
Y10_1X10/644x604 |
Y10P |
832 | 0 bytes, SOF=1 | streamed, 5025280 B |
Y8_1X8/644x604 |
GREY |
704 | 0 bytes, SOF=1 | streamed, 4252160 B |
Every byte count is exactly bytesperline * height * frames, so those are full frames. I
also ran GREY at 640x480 as a method control, since that is what my relay uses daily, to
make sure the raw-node path was not the thing failing. It passed.
So his two failing configurations stream here at the same aligned byte counts. Either his
threshold is much higher than mine or his packed path does not work at all. There is no
single number that is correct for both machines, which means a flat constant in a
clamp() cannot be right for both of us regardless of which value it holds.
What differs, none of it isolated: he is on IPU6 TGL 8086:9a19 and I am on IPU6EP ADL-P
8086:465d; he is on CSI-2 port 5 and I am on port 3, and this ADL-P only exposes ports 0-3
at all, so his port does not exist on my SoC; his link frequency is 380 or 420 MHz against my
600 MHz. I cannot test the link frequency cheaply, since it comes from the software node the
ipu-bridge builds, V4L2_CID_LINK_FREQ is read-only and there is no module parameter. For
what it is worth, a minimum-line-duration model would predict his lower link rate to be
safer rather than more fragile, so I do not think that is it, but that is an argument and not
a measurement.
Where that leaves the guard. The predicate is the reliable part, and it is already
available in try_fmt_cap(): gate on pfmt->bpp == pfmt->bpp_packed, so the SoC path stops
being restricted at all. As written the flat pixel floor silently widens a perfectly good
unpacked Y10 request for 320x240 into 392x240, because MIN_WIDTH is a clamp() rather
than a reject, and ENUM_FRAMESIZES then advertises 392 as min_width for every format.
Whether the packed side can carry a number at all is the open question: 448 holds here and
would not have helped @fildunsky.
Two more things:
- The predicate predicts who else is affected: every
bpp == bpp_packedformat takes the
MIPI path, soGREY, all*8Bayer, all*10P/*12P, plusUYVY,YUYV,RGB565and
Y16. Everything with a 16-bit container takes the SoC path. Presumably why the
visible-light cameras never showed this, since libcamera uses unpackedSBGGR10there. I
measured onlyGREY,Y10PandY10; the rest is inference from the pfmt table. - The 448 is not in the driver. No ipu6 source has a constant for a minimum line length
or byte count, and the samestridegoes down both paths, so it is enforced in the
firmware or the MIPI path hardware. That fits the failure being
wait_for_completion_timeout()on a stream_stop/close ack that never arrives, and it fits
the boundary being platform-dependent. I ruled out the iwake watermark, which uses
DIV_ROUND_UP(bytes_per_line, 2048)and floors at one page, and
IPU6_ISYS_OVERALLOC_MIN, which only inflatessizeimage.
One method warning, since it cost me an hour. media-ctl -V silently snaps to the nearest
advertised mode, the same way libcamera does without --strict-formats. Asking a driver that
advertises 644x604, 640x600 and 640x480 for 392x604 leaves the subdev at 640x600 while the
capture node accepts 392x604, and the mismatched pipeline delivers zero bytes, which looks
exactly like a real failure. Reading the subdev format back after setting it is the only way
to tell those apart.
Happy to test a candidate patch on this SP9 IR camera, or to check the UYVY/Y16
prediction, or to build extra modes at whatever geometry would be useful.
This set of changes refactors the STMicroelectronics VD55G1 and VD65G4 image sensor driver (
vd55g1) into a generic VD55G family driver (vd55g) and adds support for the VD55G0 monochrome sensor.The VD55G0 sensor is found in devices such as the Microsoft Surface Pro 9 under ACPI HID
SMO55F0. It features a native resolution of 644 x 604 pixels and differs from later variants in the family in several ways:requiring specification of generational feature differences.
to limit illuminator current and avoid tripping overcurrent protection.
Key changes:
st,vd55g0compatibility string.SMO55F0ACPI HID to the list of supported sensors on IPU6 platforms.vd55g1into genericvd55g, introduce per-chip abstractions, load firmware externally, and implement VD55G0 support.vd55g1code remnants.Tested on a Microsoft Surface Pro 9 (an Intel IPU6 platform with ACPI HID
SMO55F0) running libcamera. Validated binding schemas usingmake dt_binding_check.