Jetson agx xavier (MAX9295/MAX9296): one (AR0233) camera works, two cameras don't (CSI port too busy?)

hello!

we’re using an adlink ros cube with xavier agx SoM. there are two cameras connected to one de-serialiser (MAX9296), which in turn connects to the VI via 2 CSI ports of which only one seems to be enabled.

we can run 2 cameras in non-HDR mode ok.
in HDR (Super Exposure T1 + Linear T2) mode, one camera works ok, too, but as soon as the second camera starts sending data, both streams seem to stall (verified with v4l2-ctl) and the trace buffer shows lines like

     kworker/0:3-9137  [000] ....  1977.073275: rtcpu_vinotify_event: tstamp:62293670040 tag:CHANSEL_SHORT_FRAME channel:0x41 frame:0 vi_tstamp:62293668102 data:0x01000000

(which don’t show with a single camera.)

(even with a single camera there are messages about rtos_queue_send_from_isr_failed, but we get the desired 25fps from both cameras in that case regardless)

this seems to suggest that the frame is too large for the single CSI port between the MAX9296 and the VI and we were wondering whether and how the second port could be utilised (or why it currently isn’t). or maybe there is something else that i failed to take account of?

any hints would be really appreciated, thanks!

ps: the question seems very similar to Can't get picture simultaneously with two max9295 connect to one max9296
but in our case i cannot see mention of discarded frames in the trace buffer.

hello volker4,

this error shows the VI engine receive the frame without expectation.
short frame error means the sensor signaling coming to VI engine less than the active_h setting.

what’s the Super Exposure, is it long-exposure?
you should have two different device tree property settings for your SDR and WDR modes.
you may examine the device tree settings, especially the active region settings.
please also check Sensor Software Driver Programming Guide for reference.
thanks

hi Jerry,

thank you for your reply!

short frame error means the sensor signaling coming to VI engine less than the active_h setting.

ok. i’m assuming that since it works with one camera streaming, it is not due to the setting being wrong.

what’s the Super Exposure , is it long-exposure?

the dev guide says “Super Exposure T1 + Linear T2 sensor RAW 20−bit
output is linearized “Super Exposure” T1 − 16−bit and
following short Linear T2 − 12−bit with DLO algorithm”. does that answer your question?

you should have two different device tree property settings for your SDR and WDR modes.

right now i just have one mode defined in the device tree. do you think adding a second mode that i wouldn’t be using would change things?

you may examine the device tree settings, especially the active region settings.

since it is happy with one camera streaming, i’m assuming that those settings are ok - do you agree?

please also check Sensor Software Driver Programming Guide for reference.

i’ve been looking at that document for a while now and haven’t found a solution, is there any part in particular that you think would be helpful?

also, do you happen to know how to enable the second CSI port towards the VI? and would it make sense to use that in the first place (as in, would that give me more bandwidth?)

thanks!

hello volker4,

I don’t fully understand how this can be present in the streaming.
what’s the actual bits per pixel sending to CSI engine, 20-bit, 16-bit, or 12-bit?
what’s does the image diagram look like, let’s assume it’s 20-bit, is this a 20-bit pixel to shows 16+12-bit?
furthermore, who’s doing the T1/T2 merge?

Hello Jerry,

I don’t fully understand how this can be present in the streaming.
what’s the actual bits per pixel sending to CSI engine, 20-bit, 16-bit, or 12-bit?

i think it’s 12 bits.

what’s does the image diagram look like, let’s assume it’s 20-bit, is this a 20-bit pixel to shows 16+12-bit?

sorry, i don’t understand what you mean.

furthermore, who’s doing the T1/T2 merge?

the sensor does.

let me re-iterate my previous questions:

  • the second CSI port: is that an option at all? is it connected? and how would i go about using it?
  • do you agree that it seems to be a bandwidth problem between de-serialiser and VI?

thanks.

hello volker4,

please check this Example Digital Overlap WDR Exposure Frame (3840×2160).
it shows an example of a two-exposure DOL (Digital Overlap) multi-frame looks like in the CSI stream, and there’re several specific device tree properties need to assign to enable it.

back to your original question,

here’s documentation for Jetson Virtual Channel with GMSL Camera Framework, please refer to the diagram shows sensor connections with Aggregator. it’s able to handle dual camera in the single CSI brick.
assume you’re having correct port bindings since you’re able to enable dual camera in non-HDR mode.
and… it could bandwidth issue, and you should still back to review device tree property settings.

hello Jerry,

i’m not sure i understand the diagram. that you referenced. my understanding is that the sensor aggregates the two measurements into a 20b pixel value, then compands the result to 12b and only transfers the result, so there would be one, not two images transferred. is that wrong?

since the images are ok with just one camera streaming, i assume that all those settings like number of margin pixels, ignored lines etc are correct. would you agree?

the link to the sensor connection with aggregator does not tell me how to enable the second port. you wrote

it’s able to handle dual camera in the single CSI brick.

but also

it could bandwidth issue

which is exactly my original hypothesis and the reason why i was asking whether and how to enable the second port. how do i do that, then?

and you should still back to review device tree property settings.

yes, i’ve done that a few times now. is there any setting in particular that you think might be missing or wrong?

thanks

hello volker4,

don’t you already able to put two cameras to stream?
could you please clarify below,
for example, are you able to…

  1. stream two cameras in non-HDR mode simultaneously
  2. run single camera (i.e. cam-A) in HDR mode,
  3. please also confirm the other camera (i.e. cam-B) also able to enable in HDR mode.

don’t you already able to put two cameras to stream?

correct, but not in our HDR mode.

the reason why i’m asking about the second port is that the MAX9296 has 2 ports to the VI and my understanding is that only one of them is enabled - is that wrong?

could you please clarify below,
for example, are you able to…

stream two cameras in non-HDR mode simultaneously

yes, i mentioned that in the first post

run single camera (i.e. cam-A) in HDR mode,

yes, i mentioned that in the first post

please also confirm the other camera (i.e. cam-B) also able to enable in HDR mode.

yes, i mentioned that in the first post.

btw.: i’m looking to find out whether i can run one camera in HDR and one in non-HDR mode concurrently on the same de-serialiser. the resolution and pixel type would be all the same, though - what’s the best way of selecting an alternative mode in this case? (i’m using v4l2-ctl and gst-launch to test)

hello volker4,

please check the device tree property settings. you should have vc-id to specify these two sensors.
it should be vc-id=0 for the 1st camera sensor and vc-id=1 for the 2nd camera sensor, they could sharing the same CSI port.
note, the default mode selection logic is based on the resolution. please add boolean variable, use_sensor_mode_id into your sensor device tree for your mode selection.
thanks

please check the device tree property settings. you should have vc-id to specify these two sensors.
it should be vc-id=0 for the 1st camera sensor and vc-id=1 for the 2nd camera sensor, they could sharing the same CSI port.

current settings are like this (i assume that’s what you’re asking about, yes?):

 19         host1x {
 20                 vi@15c10000 {
 21                         num-channels = <8>;
 22                         ports {
 23                                 #address-cells = <1>;
 24                                 #size-cells = <0>;
 25                                 port@0 {
 26                                         reg = <0>;
 27                                         liar0233_vi_in0: endpoint {
 28                                                 vc-id = <0>;
 29                                                 port-index = <0>;
 30                                                 bus-width = <2>;
 31                                                 remote-endpoint = <&liar0233_csi_out0>;
 32                                         };
 33                                 };
 34                                 port@1 {
 35                                         reg = <1>;
 36                                         liar0233_vi_in1: endpoint {
 37                                                 vc-id = <1>;
 38                                                 port-index = <0>;
 39                                                 bus-width = <2>;
 40                                                 remote-endpoint = <&liar0233_csi_out1>;
 41                                         };
 42                                 };
 43                                 port@2 {
 44                                         reg = <2>;
 45                                         liar0233_vi_in2: endpoint {
 46                                                 vc-id = <0>;
 47                                                 port-index = <2>;
 48                                                 bus-width = <2>;
 49                                                 remote-endpoint = <&liar0233_csi_out2>;
 50                                         };
 51                                 };
 52  [....]

i’m adding TEGRA_CAMERA_CID_SENSOR_MODE_ID to the ioctl list and use_sensor_mode_id = "true"; to the mode, let’s see whether that makes it possible to select the mode from gst-launch and v4l2-ctl.

let me ask my original question again:

the reason why i’m asking about the second port is that the MAX9296 has 2 ports to the VI and my understanding is that only one of them is enabled - is that wrong?

if not, do you happen to know how to enable the other port?

also, since you seem to agree that this looks like a band width issue: what’s the best course of action to get data out of two cameras in HDR mode? you wrote that there were some device tree settings but didn’t mention which one…

thanks!

hello volker4,

it’s MAX9296 to output two streams, they’re carrier on CSI-A. it’s using vc-id to distinguish them (for VI) as two sources. the code-snippet shows the port binding, and it looks correct. its vc-id to share a CSI port for those SerDes sensors.
there’s sample device tree definition in Jetson Virtual Channel with GMSL Camera Framework for your reference.
note, here’re vc-id property in the gmsl-link node for each sensor, it must match the vc_id property in the sensor mode device node.

please examine SerDes Pixel Clock.
sensor driver took pixel clock to calculate the exposure and frame rate of the sensor.
it must be set correctly to avoid potential issues.
thanks

hi Jerry,

thank you. the modes all have serdes_pix_clk_hz = "833333333"; and someone claims that’s the maximum value.

would you have any suggestions?

btw we were interested whether maybe one HDR and one non-HDR stream work on the same de-serialiser. turns out that the answer is still no. also, it turns out that transitioning between modes some times gets the ISP or the VI (?) into a weird state that is only resolved with a hard reset:

kworker/0:1-815   [000] ....   760.826868: rtos_queue_peek_from_isr_failed: tstamp:24092875485 queue:0x0bcbcf78

also, i noticed that you still haven’t answered my question about the second CSI port.

how do i get CSI-B (if you want to call it that) to forward my second stream, instead of them both being virtual channels on CSI-A?

is it possible at all? in your experience, would it make sense to even try?

thanks!

hello volker4,

may I know how you validate the sensor stream.
is this HDR mode validate before, how do you confirm this HDR mode works or not?
please also check Applications Using V4L2 IOCTL Directly. please use V4L2 IOCTL to verify basic functionality during sensor bring-up.

I would like to say this is a warning message instead of an actual failure to make the camera stream malfunction.
for example, you should see the sensor timestamps still increased by checking the tstamp there,

it’s doable, if you’re having 2-lane configuration.
please set vc-id=0 for both of stream, and forward your 1st camera stream to CSI-A; 2nd stream to CSI-B.
you should define the module configuration in the device tree settings, which describes the properties of the GMSL link to which the sensor is connected.
for example,

gmsl-link {
    src-csi-port = "b";     /* Port at which sensor is connected to its serializer device. */
    dst-csi-port = "a";     /* Destination CSI port on the Jetson side, connected at deserializer. */
    serdes-csi-link = "a"; /* GMSL link sensor/serializer connected */
    Hello Jerry,

thank you for your reply!

may I know how you validate the sensor stream.
is this HDR mode validate before, how do you confirm this HDR mode works or not?

v4l2-ctl showing progress and gst-launch for visual inspection. the mode in itself works (on a single camera), when there are 2 cameras on the same de-serialiser streaming at the same time, v4l2-ctl just stalles and gstreamer shows tearing (skipped lines.

you should see the sensor timestamps still increased by checking the tstamp there,

what’s the best way of checking that?

regarding the de-serialiser connected via the other CSI port: do you mean something like the following:

     19         host1x {                                                                                                                                      
     20                 vi@15c10000 {                                                                                                                         
     21                         num-channels = <8>;                                                                                                           
     22                         ports {                                                                                                                       
     23                                 #address-cells = <1>;                                                                                                 
     24                                 #size-cells = <0>;                                                                                                    
     25                                 port@0 {                                                                                                              
     26                                         reg = <0>;                                                                                                    
     27                                         liar0233_vi_in0: endpoint {                                                                                   
     28                                                 vc-id = <0>;                                                                                          
     29                                                 port-index = <0>;                                                                                     
     30                                                 bus-width = <2>;                                                                                      
     31                                                 remote-endpoint = <&liar0233_csi_out0>;                                                               
     32                                         };                                                                                                            
     33                                 };                                                                                                                    
     34                                 port@1 {                                                                                                              
     35                                         reg = <1>;                                                                                                    
     36                                         liar0233_vi_in1: endpoint {                                                                                   
     37                                                 vc-id = <0>;                                                                                          
     38                                                 port-index = <1>;                                                                                     
     39                                                 bus-width = <2>;                                                                                      
     40                                                 remote-endpoint = <&liar0233_csi_out1>;                                                               
     41                                         };                                                                                                            
     42                                 };
    [....]
    100                 nvcsi@15a00000 {                                                                                                                      
    101                         num-channels = <8>;                                                                                                           
    102                         #address-cells = <1>;                                                                                                         
    103                         #size-cells = <0>;                                                                                                            
    104                         channel@0 {                                                                                                                   
    105                                 reg = <0>;                                                                                                            
    106                                 ports {                                                                                                               
    107                                         #address-cells = <1>;                                                                                         
    108                                         #size-cells = <0>;                                                                                            
    109                                         port@0 {                                                                                                      
    110                                                 reg = <0>;                                                                                            
    111                                                 liar0233_csi_in0: endpoint@0 {                                                                        
    112                                                         port-index = <0>;                                                                             
    113                                                         bus-width = <2>;                                                                              
    114                                                         remote-endpoint = <&liar0233_ar0233_out0>;                                                    
    115                                                 };                                                                                                    
    116                                         };                                                                                                            
    117                                         port@1 {                                                                                                      
    118                                                 reg = <1>;                                                                                            
    119                                                 liar0233_csi_out0: endpoint@1 {                                                                       
    120                                                         remote-endpoint = <&liar0233_vi_in0>;                                                         
    121                                                 };                                                                                                    
    122                                         };                                                                                                            
    123                                 };                                                                                                                    
    124                         };                                                                                                                            
    125                         channel@1 {                                                                                                                   
    126                                 reg = <1>;                                                                                                            
    127                                 ports {                                                                                                               
    128                                         #address-cells = <1>;                                                                                         
    129                                         #size-cells = <0>;                                                                                            
    130                                         port@0 {                                                                                                      
    131                                                 reg = <0>;                                                                                            
    132                                                 liar0233_csi_in1: endpoint@2 {                                                                        
    133                                                         port-index = <1>;                                                                             
    134                                                         bus-width = <2>;                                                                              
    135                                                         remote-endpoint = <&liar0233_ar0233_out1>;                                                    
    136                                                 };                                                                                                    
    137                                         };                                                                                                            
    138                                         port@1 {                                                                                                      
    139                                                 reg = <1>;                                                                                            
    140                                                 liar0233_csi_out1: endpoint@3 {                                                                       
    141                                                         remote-endpoint = <&liar0233_vi_in1>;                                                         
    142                                                 };                                                                                                    
    143                                         };                                                                                                            
    144                                 };                                                                                                                    
    145                         };100                 nvcsi@15a00000 {                                                                                                                      
    101                         num-channels = <8>;                                                                                                           
    102                         #address-cells = <1>;                                                                                                         
    103                         #size-cells = <0>;                                                                                                            
    104                         channel@0 {                                                                                                                   
    105                                 reg = <0>;                                                                                                            
    106                                 ports {                                                                                                               
    107                                         #address-cells = <1>;                                                                                         
    108                                         #size-cells = <0>;                                                                                            
    109                                         port@0 {                                                                                                      
    110                                                 reg = <0>;                                                                                            
    111                                                 liar0233_csi_in0: endpoint@0 {                                                                        
    112                                                         port-index = <0>;                                                                             
    113                                                         bus-width = <2>;                                                                              
    114                                                         remote-endpoint = <&liar0233_ar0233_out0>;                                                    
    115                                                 };                                                                                                    
    116                                         };                                                                                                            
    117                                         port@1 {                                                                                                      
    118                                                 reg = <1>;                                                                                            
    119                                                 liar0233_csi_out0: endpoint@1 {                                                                       
    120                                                         remote-endpoint = <&liar0233_vi_in0>;                                                         
    121                                                 };                                                                                                    
    122                                         };                                                                                                            
    123                                 };                                                                                                                    
    124                         };                                                                                                                            
    125                         channel@1 {                                                                                                                   
    126                                 reg = <1>;                                                                                                            
    127                                 ports {                                                                                                               
    128                                         #address-cells = <1>;                                                                                         
    129                                         #size-cells = <0>;                                                                                            
    130                                         port@0 {                                                                                                      
    131                                                 reg = <0>;                                                                                            
    132                                                 liar0233_csi_in1: endpoint@2 {                                                                        
    133                                                         port-index = <1>;                                                                             
    134                                                         bus-width = <2>;                                                                              
    135                                                         remote-endpoint = <&liar0233_ar0233_out1>;                                                    
    136                                                 };                                                                                                    
    137                                         };                                                                                                            
    138                                         port@1 {                                                                                                      
    139                                                 reg = <1>;                                                                                            
    140                                                 liar0233_csi_out1: endpoint@3 {                                                                       
    141                                                         remote-endpoint = <&liar0233_vi_in1>;                                                         
    142                                                 };                                                                                                    
    143                                         };                                                                                                            
    144                                 }; 
    [...]                                                                                                       
    275         i2c@c240000 {                                                                                                                                 
    276                 tca9546@70 {                                                                                                                          
    277                         i2c@0 {                                                                                                                       
    278                         ar0233_a@1b {                                                                                                                 
    279                                 reg = <0x1b>;                                                                                                         
    280                                 reg_mux = <0>;                                                                                                        
    281                                                                                                                                                       
    282                                 #undef TEGRA_SINTERFACE                                                                                               
    283                                 #define TEGRA_SINTERFACE "serial_a"                                                                                   
    284                                 #undef VC_ID                                                                                                          
    285                                 #define VC_ID "0"                                                                                                     
    286                                 #include "ar0233-mode-settings.dtsi"                                                                                  
    287                                                                                                                                                       
    288                                 ports {                                                                                                               
    289                                         #address-cells = <1>;                                                                                         
    290                                         #size-cells = <0>;                                                                                            
    291                                         port@0 {                                                                                                      
    292                                                 reg = <0>;                                                                                            
    293                                                 liar0233_ar0233_out0: endpoint {                                                                      
    294                                                         vc-id = <0>;                                                                                  
    295                                                         port-index = <0>;                                                                             
    296                                                         bus-width = <2>;                                                                              
    297                                                         remote-endpoint = <&liar0233_csi_in0>;                                                        
    298                                                         };                                                                                            
    299                                                 };                                                                                                    
    300                                         };                                                                                                            
    301                                 gmsl-link {                                                                                                           
    302                                         src-csi-port = "b";                                                                                           
    303                                         dst-csi-port = "a";                                                                                           
    304                                         serdes-csi-link = "a";                                                                                        
    305                                         csi-mode = "1x4";                                                                                             
    306                                         st-vc = <0>;                                                                                                  
    307                                         vc-id = <0>;                                                                                                  
    308                                         num-lanes = <2>;                                                                                              
    309                                         streams = "ued-u1", "raw12";                                                                                  
    310                                         };                                                                                                            
    311                                 };                                                                                                                    
    312                                                                                                                                                       
    313                         ar0233_b@1c {                                                                                                                 
    314                                 reg = <0x1c>;                                                                                                         
    315                                 reg_mux = <0>;                                                                                                        
    316                                                                                                                                                       
    317                                 #undef TEGRA_SINTERFACE                                                                                               
    318                                 #define TEGRA_SINTERFACE "serial_b"                                                                                   
    319                                 #undef VC_ID                                                                                                          
    320                                 #define VC_ID "0"                                                                                                     
    321                                 #include "ar0233-mode-settings.dtsi"                                                                                  
    322                                                                                                                                                       
    323                                 ports {                                                                                                               
    324                                         #address-cells = <1>;                                                                                         
    325                                         #size-cells = <0>;                                                                                            
    326                                         port@0 {                                                                                                      
    327                                                 reg = <0>;                                                                                            
    328                                                 liar0233_ar0233_out1: endpoint {                                                                      
    329                                                                 vc-id = <0>;                                                                                  
    330                                                         port-index = <0>;                                                                             
    331                                                         bus-width = <2>;                                                                              
    332                                                         remote-endpoint = <&liar0233_csi_in1>;                                                        
    333                                                         };                                                                                            
    334                                                 };                                                                                                    
    335                                         };                                                                                                            
    336                                 gmsl-link {                                                                                                           
    337                                         src-csi-port = "b";                                                                                           
    338                                         dst-csi-port = "a";                                                                                           
    339                                         serdes-csi-link = "b";                                                                                        
    340                                         csi-mode = "1x4";                                                                                             
    341                                         st-vc = <0>;                                                                                                  
    342                                         vc-id = <0>;                                                                                                  
    343                                         num-lanes = <2>;                                                                                              
    344                                         streams = "ued-u1", "raw12";                                                                                  
    345                                         };                                                                                                            
    346                                 };                                                                                                                    
    347                         };         

with these settings, the behaviour seems to be exactly the same as before, sadly.
thanks!

could you please share the video clips to show the abnormal results.
thanks

this looks incorrect,
you would like to have 1st cam to CSI-A and 2nd cam to CSI-B,
so, you should have configure dserializer to Jetson CSI port connection, (i.e. dst-csi-port) accordingly.

for your reference,
here’s an abstract connection diagram,

[Sensor] ---> [Serializer] ---> [Deserializer] ---> [Jetson]

Hi Jerry,
This doesn’t fix the problem. The MAX9296 driver seems to be incomplete. If dst-csi-port is set to ‘b’, the driver does not configure pipes correctly. It looks like this is is due to the all of the pipes being initialised with MAX9296_CSI_CTRL_1 in max9296_pipes_reset(), and the switch statement in max9296_get_available_pipe() only looking at these pipes if dst_csi_port == GMSL_CSI_PORT_A

Both excerpts from max9296.c:

 175 static void max9296_pipes_reset(struct max9296 *priv)
 176 {
 177         /*
 178          * This is default pipes combination. add more mappings
 179          * for other combinations and requirements.
 180          */
 181         struct pipe_ctx pipe_defaults[] = {
 182                 {MAX9296_PIPE_X, GMSL_CSI_DT_RAW_12,
 183                         MAX9296_CSI_CTRL_1, 0, MAX9296_INVAL_ST_ID},
 184                 {MAX9296_PIPE_Y, GMSL_CSI_DT_RAW_12,
 185                         MAX9296_CSI_CTRL_1, 0, MAX9296_INVAL_ST_ID},
 186                 {MAX9296_PIPE_Z, GMSL_CSI_DT_EMBED,
 187                         MAX9296_CSI_CTRL_1, 0, MAX9296_INVAL_ST_ID},
 188                 {MAX9296_PIPE_U, GMSL_CSI_DT_EMBED,
 189                         MAX9296_CSI_CTRL_1, 0, MAX9296_INVAL_ST_ID}
 190         };
 191 
 192         /*
 193          * Add DT props for num-streams and stream sequence, and based on that
 194          * set the appropriate pipes defaults.
 195          * For now default it supports "2 RAW12 and 2 EMBED" 1:1 mappings.
 196          */
 197         memcpy(priv->pipe, pipe_defaults, sizeof(pipe_defaults));
 198 }
 518 static int max9296_get_available_pipe(struct device *dev,
 519                                 u32 st_data_type, u32 dst_csi_port)
 520 {
 521         struct max9296 *priv = dev_get_drvdata(dev);
 522         int i;
 523 
 524         for (i = 0; i < MAX9296_MAX_PIPES; i++) {
 525                 /*
 526                  * TODO: Enable a pipe for multi stream configuration having
 527                  * similar stream data type. For now use st_count as a flag
 528                  * for 1 to 1 mapping in pipe and stream data type, same can
 529                  * be extended as count for many to 1 mapping. Would also need
 530                  * few more checks such as input stream id select, dst port etc.
 531                  */
 532                 if ((priv->pipe[i].dt_type == st_data_type) &&
 533                         ((dst_csi_port == GMSL_CSI_PORT_A) ?
 534                                 (priv->pipe[i].dst_csi_ctrl ==
 535                                         MAX9296_CSI_CTRL_0) ||
 536                                 (priv->pipe[i].dst_csi_ctrl ==
 537                                         MAX9296_CSI_CTRL_1) :
 538                                 (priv->pipe[i].dst_csi_ctrl ==
 539                                         MAX9296_CSI_CTRL_2) ||
 540                                 (priv->pipe[i].dst_csi_ctrl ==
 541                                         MAX9296_CSI_CTRL_3)) &&
 542                         (!priv->pipe[i].st_count))
 543                         break;
 544         }
 545 
 546         if (i == MAX9296_MAX_PIPES) {
 547                 dev_err(dev, "%s: all pipes are busy\n", __func__);
 548                 return -ENOMEM;
 549         }
 550 
 551         return i;
 552 }