MSELECT error detected! status=0x100

Hey,

I am using a TK1 board, but it’s of another brand (Toradex), using their L4T based Linux image with kernel 3.10.
When I plug in a custom PCIe card and try to set config registers in the card using /dev/mem, and use ethernet, I get that error above. The serial debug console is flooded with that message, until the kernel decides to auto-reboot after a while.

I had the very same custom PCIe card in a TK1 board of yet another maker, and did not see that problem. Also using an L4T based image.

I have seen threads on here of similar error messages, but the surrounding scenarios were quite different.

Basically I’m looking now everywhere I can for clues as to what could be wrong (of course also with support at the manufacturer’s site, so far not fruitful).

Any ideas?

I can’t answer, but a starting question might help. Does the driver (I’m assuming you wrote it) use physical address or virtual address? Is DMA involved? Prior to setting registers what do you see from “sudo lspci -vvv”? In the TK1 I think the driver probably had to use physical address.

If there is any way to post a sample of those errors (which should be possible even on a failing system with serial console) it would be useful to see what goes on just before the error and at least the start of errors (and if possible maybe what is shown just at the point of reboot).

Thanks for your reply,

I am not a driver dev and only know one or the other thing about the subject because I recently tried to debug a driver that seemed to almost work & read up on this or that.

Hence for this I am only using the /dev/mem character driver to write to some memory mapped registers which the card exposes.
For that, I took a BAR address as listed for the PCIe device and added an offset to it which the HW guy told me. The sum I hand to mmap(…) to map a 1MB region or so. This works on a different board.

The card houses a FPGA and exposes register blocks only for configuration. The bulk data gets output by the card’s LAN output. (it’s not quite a small mini PCIe card, it’s a larger card with an adapter cable)

No DMA is involved here on this PCIe.

Upon the suggestion of the manufacturer, I installed a Linux image now that’s not L4T based, with a newer kernel. This yields the interesting result that whenever the PCIe card is in, the LAN does not work.
Does this hint at something that happens with both Linux images, but are manifesting in different ways?

Since when the PCIe card is on, I get no functioning LAN on the TK1 board, I copied this from serial console - apologies if other messages are in there in between ;)
Also, since LAN does not work, I can’t replicate th earlier MSELECT error scenario right now, but if it makes sense, I’ll install the L4T based image again and do it with that later.
(btw, the board manufacturer did not have dysfunctional LAN when he inserted some random PCIe card)

Interesting: I don’t see the BAR address that I saw for the other TK1 based board and that I was using (For the Xilinx one, something with 0x328… it used to be). Should they be the same on all TK1 boards, for the same PCIe device? (sorry, no idea of how those numbers get created).

lspci -vvv
00:01.0 PCI bridge: NVIDIA Corporation TegraK1 PCIe x4 Bridge (rev a1) (prog-if 00 [Normal decode])
        Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 64 bytes
        Interrupt: pin A routed to IRQ 396
        Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
        I/O behind bridge: 0000f000-00000fff [empty]
        Memory behind bridge: 13000000-13bfffff 
        Prefetchable memory behind bridge: 0000000020000000-0000000020ffffff 
        Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
        BridgeCtl: Parity+ SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
                PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
        Capabilities: [40] Subsystem: NVIDIA Corporation TegraK1 PCIe x4 Bridge
        Capabilities: [48] Power Management version 3
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold+)
                Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [50] MSI: Enable+ Count=1/2 Maskable- 64bit+
                Address: 00000000ae3ab000  Data: 0000
        Capabilities: [60] HyperTransport: MSI Mapping Enable- Fixed-
                Mapping Address Base: 00000000fee00000
        Capabilities: [80] Express (v2) Root Port (Slot+), MSI 00
                DevCap: MaxPayload 128 bytes, PhantFunc 0
                        ExtTag+ RBE+
                DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
                        RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+
                        MaxPayload 128 bytes, MaxReadReq 512 bytes
                DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
                LnkCap: Port #0, Speed 5GT/s, Width x2, ASPM L0s, Exit Latency L0s <512ns
                        ClockPM- Surprise- LLActRep+ BwNot+ ASPMOptComp-
                LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk-
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive+ BWMgmt- ABWMgmt-
                SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
                        Slot #0, PowerLimit 0.000W; Interlock- NoCompl-
                SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
                        Control: AttnInd Off, PwrInd On, Power- Interlock-
                SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-
                        Changed: MRL- PresDet+ LinkState+
                RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna+ CRSVisible-
                RootCap: CRSVisible-
                RootSta: PME ReqID 0000, PMEStatus- PMEPending-
                DevCap2: Completion Timeout: Range AB, TimeoutDis+, LTR-, OBFF Not Supported ARIFwd-
                         AtomicOpsCap: Routing- 32bit- 64bit- 128bitCAS-
                DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled ARIFwd-
                         AtomicOpsCtl: ReqEn- EgressBlck-
                LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-
                         Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
                         Compliance De-emphasis: -6dB
                LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1-
                         EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
        Kernel driver in use: pcieport

00:02.0 PCI bridge: NVIDIA Corporation TegraK1 PCIe x1 Bridge (rev a1) (prog-if 00 [Normal decode])
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 64 bytes
        Interrupt: pin A routed to IRQ 397
        Bus: primary=00, secondary=02, subordinate=02, sec-latency=0
        I/O behind bridge: 00001000-00001fff 
        Memory behind bridge: 13c00000-13cfffff 
        Prefetchable memory behind bridge: 00000000fff00000-00000000000fffff [empty]
        Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
        BridgeCtl: Parity+ SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
                PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
        Capabilities: [40] Subsystem: NVIDIA Corporation TegraK1 PCIe x1 Bridge
        Capabilities: [48] Power Management version 3
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold+)
                Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [50] MSI: Enable+ Count=1/2 Maskable- 64bit+
                Address: 00000000ae3ab000  Data: 0001
        Capabilities: [60] HyperTransport: MSI Mapping Enable- Fixed-
                Mapping Address Base: 00000000fee00000
        Capabilities: [80] Express (v2) Root Port (Slot+), MSI 00
                DevCap: MaxPayload 128 bytes, PhantFunc 0
                        ExtTag+ RBE+
                DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
                        RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+
                        MaxPayload 128 bytes, MaxReadReq 512 bytes
                DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
                LnkCap: Port #1, Speed 5GT/s, Width x1, ASPM L0s, Exit Latency L0s <512ns
                        ClockPM- Surprise- LLActRep+ BwNot+ ASPMOptComp-
                LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk-
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive+ BWMgmt- ABWMgmt-
                SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
                        Slot #0, PowerLimit 0.000W; Interlock- NoCompl-
                SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
                        Control: AttnInd Off, PwrInd On, Power- Interlock-
                SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-
                        Changed: MRL- PresDet+ LinkState+
                RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna+ CRSVisible-
                RootCap: CRSVisible-
                RootSta: PME ReqID 0000, PMEStatus- PMEPending-
                DevCap2: Completion Timeout: Range AB, TimeoutDis+, LTR-, OBFF Not Supported ARIFwd-
                         AtomicOpsCap: Routing- 32bit- 64bit- 128bitCAS-
                DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled ARIFwd-
                         AtomicOpsCtl: ReqEn- EgressBlck-
                LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-
                         Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
                         Compliance De-emphasis: -6dB
                LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1-
                         EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
        Kernel driver in use: pcieport

01:00.0 RAM memory: Xilinx Corporation Device 7011
        Subsystem: Xilinx Corporation Device 0007
        Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Interrupt: pin A routed to IRQ 255
        Region 0: Memory at 13000000 (64-bit, non-prefetchable) [disabled] 
        Region 2: Memory at 13800000 (64-bit, non-prefetchable) [disabled] 
        Region 4: Memory at 20000000 (64-bit, prefetchable) [disabled] 
        Capabilities: [40] Power Management version 3
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold-)
                Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [48] MSI: Enable- Count=1/1 Maskable- 64bit+
                Address: 0000000000000000  Data: 0000
        Capabilities: [60] Express (v2) Endpoint, MSI 00
                DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s <64ns, L1 unlimited
                        ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 0.000W
                DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
                        RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+
                        MaxPayload 128 bytes, MaxReadReq 512 bytes
                DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
                LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s, Exit Latency L0s unlimited
                        ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp-
                LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk-
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
                DevCap2: Completion Timeout: Range B, TimeoutDis-, LTR-, OBFF Not Supported
                         AtomicOpsCap: 32bit- 64bit- 128bitCAS-
                DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
                         AtomicOpsCtl: ReqEn-
                LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-
                         Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
                         Compliance De-emphasis: -6dB
                LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1-
                         EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
        Capabilities: [100 v1] Device Serial Number 00-00-00-00-00-00-00-00

02:00.0 Ethernet controller: Intel Corporation I210 Gigabit Network Connection (rev 03)
        Subsystem: Intel Corporation I210 Gigabit Network Connection
        Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 64 bytes
        Interrupt: pin A routed to IRQ 22
        Region 0: Memory at 13c00000 (32-bit, non-prefetchable) 
        Region 2: I/O ports at 1000 [disabled] 
        Region 3: Memory at 13c20000 (32-bit, non-prefetchable) 
        Capabilities: [40] Power Management version 3
                Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
                Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=1 PME-
        Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
                Address: 0000000000000000  Data: 0000
                Masking: 00000000  Pending: 00000000
        Capabilities: [70] MSI-X: Enable+ Count=5 Masked-
                Vector table: BAR=3 offset=00000000
                PBA: BAR=3 offset=00002000
        Capabilities: [a0] Express (v2) Endpoint, MSI 00
                DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us
                        ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ SlotPowerLimit 0.000W
                DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+
                        RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ FLReset-
                        MaxPayload 128 bytes, MaxReadReq 512 bytes
                DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend-
                LnkCap: Port #1, Speed 2.5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s unlimited, L1 <16us
                        ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
                LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk-
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
                DevCap2: Completion Timeout: Range ABCD, TimeoutDis+, LTR-, OBFF Not Supported
                         AtomicOpsCap: 32bit- 64bit- 128bitCAS-
                DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
                         AtomicOpsCtl: ReqEn-
                LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-
                         Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
                         Compliance De-emphasis: -6dB
                LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
                         EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
        Capabilities: [100 v2] Advanced Error Reporting
                UESta:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                UEMsk:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
                CESta:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
                CEMsk:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
                AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn-
                        MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
                HeaderLog: 00000000 00000000 00000000 00000000
        Capabilities: [140 v1] Device Serial Number 00-a0-c9-ff-ff-00-00-00
        Capabilities: [1a0 v1] Transaction Processing Hints
                Device specific mode supported
                Steering table in TPH capability structure
        Kernel driver in use: igb

I can’t give a good answer. However, there are some specific changes to the kernel for using with the Tegra SoC, and so I’d be very suspicious of the kernel. Someone else will have to answer if the BAR address use is valid…there are sometimes different configurations which change mapping between physical and virtual (and I don’t know if that applies to your case). With what you’ve posted someone should be able to help now.

Oh, you mean, run-off-the-mill kernels can’t actually run on the Tegra? I.e. if the board manufactuter offers an alternate Linux to the L4T-based one, hey would have needed to replicate those?
If it’s only about CUDA related stuff - that’s explicitly left out of that alternate kernel.

But it seems nobody here has an idea, alas.

There is some out of tree content that stock kernels don’t ship with. I don’t know if mainstream “cannot” be used, but there would be some features mainstream has not adopted. At minimum you would lose some functionality with mainstream unless you did some porting. Typically if you use kernel modules from outside and compiled against this kernel you would get 100% compatibility, but if you used the existing “/proc/config.gz” on the alternate kernel, then likely you would have some compile errors.

The differences are typically due to the SoC and not due to to CUDA. The GPU drivers are in binary format, and are bound to a specific X11 ABI, but I’m not sure if similar kernels would or would not be able to use the binary GPU drivers…probably similar kernel versions with the same configuration could, but it isn’t guaranteed.

The main change if you changed a carrier board or used a custom board is in the device tree. The device tree itself is a bit like a blueprint to drivers doing hardware setup. A manufacturer-provided board support package would provide an alternative device tree, and if the device tree is compatible with another kernel, then that kernel will probably work. Some of the customizations though for the NVIDIA kernel are device tree related.

Is the kernel you are using from L4T? Is this kernel a stock configuration?

Interesting. That board manufacturer also has aboard with a different SoC, and iMX6, and there, for my FPGA card, the PCIe also does not work (different failure mode, though), but for them, mini-PCIe addon cards such as network function - as do with their TK1 based board, they say.
So it’s apparently only (from what I can test) the combination of their boards and that one specific PCIe device (the Xilinx artix7 board) where there are problems - on another maker’s TK1 board, the Xilinx PCIe thing works, as already mentioned.
I wonder what’s different between the FPGA board (which uses a Xilinx-provided standard block for PCIe inside the FPGA) and the shrink-wrap PCIe cards they tried.

As for my kernel:
I tried two different ones.
A) Their L4T-based one(but somehow extended/modified for their stuff, well I don’t know whether they changed thekernel per se or merely something else in the Linux + uboot installer image)
and B) their mainlie kernel based one, where only open source drivers are used, and things only available as binary from NVIDIA, left out, if I understood it correctly.

Their L4T based image gets this “MSELECT” error this thread is named after.
Their mainline kernel based image boots without working LAN if the FPGA board is connected & operational, but with LAN if it’s off, but does not produce the “MSELECT” error.

I don’t know the details, though someone else may know…some issues with FPGAs are solved by making sure the FPGA is fully booted prior to booting the TK1. Or, alternatively (and I don’t know the details of this either), the TK1’s kernel is modified to wait before enumerating PCIe devices. Hot plug PCIe is not supported, and so one fix or the other is required if the PCIe device is not fully set up at the time of boot of the TK1.

Can someone exlpain what MSELECT actually is, in the Tegra K1 SoC?

I found, in the L4T source tree under drivers…platform…nvidia: hier_ictlr.c, the interrupt handler is the place where the “MSELECT error…” message is printed. So I found the tegra_hier_ictlr struct and it’s member mselect_base - which is defined somewhere as an address.

So I look in the tech ref man, found section 32.0, PCI-E, there is a block diagram with MSelect, between CPU complex and PCIe / other stuff.
There is some register space address defined for it in the manual, and it’s referenced in a few places, but I saw no explanation as to what MSelect is exactly. Seems to be some sort of clock mux switch?

It might help to actually know what an “MSelect error” means, and what the source of such an error could be.


As for FPGA booting, yes, I always program the FPGA first, then boot the TK1 Linux.

Someone else may need to give more details, but this centers on how parts of the Tegra chips integrate with the memory controller in a manner which is different than that of a desktop PC. From the “Tegra Linux Driver Package Development Guide”:

PCIe: Tegra TX1 does not have any path from AHB-DMA or APB-
DMA engines to PCIe IP as PCIe is connected directly to MSELECT and
AHB-DMA and APB DMA engines only interact with IPs connected to
respective AHB and APB buses. So it is not possible to use either AHB
or APB engines for PCIe.

Basically there are parts of the Tegra series which integrate directly to the memory controller, but in a PC this is not the case. DMA over PCI differs between the two platforms because of this (and this is an example why a desktop PC video driver won’t work…they are intended for PCIe, but the integrated GPU is part of the memory controller…and vice versa, the integrated GPU driver won’t work with PCIe cards due to needing PCI interface). If you see the error, then it implies DMA code may need to be adjusted to go through this memory controller.

I can’t give any useful advice on the actual adjustment, someone else will need to comment.