Announcement

Collapse
No announcement yet.

System76 partial power-on, VDD1.05 out fail

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    System76 partial power-on, VDD1.05 out fail

    Laptop:
    Hello, I have a System76 Gazelle laptop (model gaze14) which now only partially powers on and does not boot properly.

    History:
    This laptop has served me well for a few years. It's always been a bit noisy with the fans and recently became much more so on both fans, where just opening a vim/notepad session would cause the fans to spin up loudly and make a grating sound, like the ball bearing were mechanically wearing out. So after blowing the dust out of the fans out didn't help, I ordered replacement fans and heat sink component from System76 which were sold as a unit (1 CPU fan, 1 GPU fan, 1 connected Cu heatpipe/sink assembly). I opened the laptop up, swapped out the old heatsink and fans and installed the new ones. I also opted to try using Innovation Cooling Graphite Thermal Pad instead of thermal grease as I had done in the past. Knowing that the graphite pads were conductive, I made sure to carefully cut the pads to size after measuring the dimensions of CPU and GPU with calipers and not to let the pads touch other components, including while the laptop was unpowered. After happily re-assembling the laptop with the new gear ready to give it a spin, I pressed the power button, and it didn't boot. Yes, so it appears I have somehow caused damage during my fan/heatsink swap episode, perhaps a thermal pad induced short despite careful handling and placement; otherwise, I just got very unlucky that it happened to coincidentally fail after I worked on it.

    Boot Observation:
    - The laptop shows an LED indicator properly for AC_IN which is orange. When the lid is opened and the power button is pressed, the AC LED indicator turns green for about 3 seconds, then turns back to orange again and nothing happens: no LCD display change, no fans, nothing. This is repeatable.
    - The laptop also shows a separate battery LED indicator properly (orange), and when it is charging (green).

    Problem appears to be the motherboard:
    - The problem is the same with or without RAM present, with or without M2 SSD storage present. Removing the CMOS battery temporarily and re-connecting does not resolve problem or change behavior.
    - Removing the motherboard from the laptop, removing RAM and storage and disconnecting any extra circuit connector wiring to eliminate other possibilities (ie: disconnect CCD cable, USB board cable, touchpad cable, keyboard cable, CPU/GPU fans) and using only AC in with no battery present still results in same power on problem behavior.

    Hardware Details:
    - This computer appears to be a rebrand based on Clevo hardware, with a "Clevo CO Code" on the back of "NH58RA". I was able to find a service manual with schematics for this board. The part number on the motherboard is: 6-77-NH50RA00-D02B-1E#10. I'm not sure if it's appropriate to post the PDF directly or link to it (if I can still find it) since it's not directly from the Clevo website that I obtained it from, so I'll leave that out for now.

    Hardware troubleshooting:
    - Initial inspection of the board doesn't reveal any obvious physical damage anywhere
    - Power jack, mosfets near power jack and random multimeter testing of various capacitors I can readily identify on the board don't appear to show any shorts to ground.
    - The service manual does show the following power on steps, shown partially here, in order they are shown in the service manual, stopping at the first line item where things don't appear to be healthy. I used the circuit schematics to locate a component where I could test voltage of each respective item and identified the first issue which appears to be output of VDD1.05 which in this board's case, appears to be near 0V:

    Code:
    VDD3   ~3.4V
    SLP_SUS# ~3.4V
    3.3VA   ~3.4V
    1.8VA   ~1.86V
    PWR_BTN# ~3.4V
    VDD1.05  ~0.01V  <--- Looks like the first power on line item issue
    I checked a few line items further in the power on process after VDD1.05. The next line item is RSMRST and that appears to be normal. After that is DD_ON, and that's essentially also 0V. I suspect quite a few power on steps after that are also not normal, so I've started to try and locate the source of the problem where is it first seen, in the VDD1.05 circuit output.

    It looks to me like the PU21 chip (see pic) should be generating VDD1.05 on pins 6, 19, 20. I measure voltage on either side of inductor PL16 and get near 0V. I measure voltage at other components in the schematics where VDD1.05 is shown and also get near 0V. PU21 chip does have VIN on pins 2, 3, 4, 5 of ~19.6V. I did look briefly at the other pins on PU21 which seemed either ok or inconclusive (I didn't record notes on the other pins so can't comment further than that at the moment).

    Not sure exactly what to do for next steps in troubleshooting. I do not have an o-scope at the moment, just a multi-meter.

    Thoughts:
    - I suppose I might have possibly shorted the CPU/GPU or some components around it by trying out the graphite thermal pad despite conscious and careful handling and installation. I'm hoping I'm not to blame for destroying my own board, but who knows. Given the laptop had been running hot with the fans spinning high and also sounding like they were starting to die which prompted me to order replacement fan components in the first place, I also wouldn't be too surprised if this is completely unrelated to the thermal pads and some component just coincidentally died from overheating and the corresponding lifetime shortening in the weeks prior.

    Any suggestions for further troubleshooting would be much appreciated. I don't yet know how to test the CPU and GPU for shorts or problems, but I'll be researching that soon to try and figure it out if I don't get anywhere with finding root cause for the lack of VDD1.05 signal.

    Thank you!
    John
    Attached Files

    #2
    Re: System76 partial power-on, VDD1.05 out fail

    After reviewing the forum further, it appears I can upload the service manual for this MB with schematics. Attached.
    Attached Files

    Comment


      #3
      Re: System76 partial power-on, VDD1.05 out fail

      1) measure the voltage of the EN (enable) pin of the VDD1.05 regulator. Post the measurement. This is to confirm that the logic board is requesting for this CPU power rail to turn ON.

      2) Remove all power -> meter in resistance mode -> check the resistance to ground of the VDD1.05 output (checking at the inductor / coil is ok).

      This regulator may be shutting off due to an excessive current draw. Perhaps due to a shorted CPU / PCH.

      Comment


        #4
        Re: System76 partial power-on, VDD1.05 out fail

        Thanks for the suggestions @mon2!

        1) EN pin 11 reads about 0.48V, and fluctuates around a little bit. This measures about the same immediately after providing AC power, and also after pressing the power-on switch.

        2) With no power, both sides of PL16 inductor which comes from pins 16, 19 and 20, measure about 40 Ohms to ground.

        Since this problem started, the white color power-on indicator LED mounted next to the power button at the top right of the keyboard area has never come on -- the only visual indication something is happening when I press the power button is that the power LED on the front edge of the machine which indicates AC_IN changes from orange to green for about 3 seconds, then back to orange.

        However, when I was repeating the measurements above to ensure my measurements were correct and I pressed the power on button after supplying AC, for the first time since this issue started, the white LED indicator beside the power switch came on and the AC_IN LED indicator stayed green rather than turning back to orange in 3 seconds. I immediately thought power may be now going to the CPU and GPU and that without the heatsinks mounted, they might quickly overheat and cause some damage (is that plausible?), so I unplugged power quickly to think about what to do next. When I plugged it back in to re-try, behavior had reverted again to the LED for AC_IN indicator just goes back to orange in 3 seconds and the white power-on LED by top right of the keyboard never coming on. I tried several times to repeat without luck. Not sure why it behaved differently on that one try.

        In any case, I looked for docs for the PU21 (SY8288RAC) to see what the required enable voltage is, but am not finding a datasheet. Probably ~0.4V is not sufficient for enable though. Looks like this enable signal for PU21 on pin 11 should probably be coming from 1.8VA_PGD input, so I'll see what I can figure out there next.
        Last edited by johnnylotto; 09-18-2022, 01:22 PM.

        Comment


          #5
          Re: System76 partial power-on, VDD1.05 out fail

          This is a very detailed explanation of the issue. I agree with you to trace the missing enable back to see why it is missing. The method to confirm whether the enable is always low by checking voltages while plugging the charger (or pressing the power button in other cases) is a clever step.

          Is the resistance to GND of the 3.3VA healthy?

          Good luck.
          FairRepair on YouTube

          Comment


            #6
            Re: System76 partial power-on, VDD1.05 out fail

            Hi @Sephir0th, thanks! This is my first time looking at a motherboard in detail and I'm finding this quite challenging to determine to failure and solution! The 3.3VA output appears to have about 1.86 kOhm resistance to ground as measured from PJ6 and also one side of C356, so that seems to be ok.

            TLDR Summary:
            1) PC207 is missing from the board, likely due to an initial forceful fan connector removal
            2) PR225 is giving the same voltages on each end of the resistor when the board is powered which seems like it is in contradiction with the schematic. When the power is off, the resistor shows expected resistance and so doesn't seem to be shorted itself. I don't understand what is going on here and if this indicates another problem somewhere.

            Details:

            Checking PU19 where 1.8VA_PGD originates from:
            The 1.8VA diagram of PU19 (attached) seems to check out ok with the 1.8 VA output ok (measures: 1.86V at PL14), VIN is good (19.7V at PJ41), EN/DSG seems ok depending on threshold (1.7V at PJ38), and perhaps most importantly, PGOOD on pin 1 which produces the 1.8VA_PGD signal that goes to PU21 is 1.85V.

            Strange resistor reading on PU19:
            The schematic diagram shows PR225 resistor connecting to the PGOOD pin on one side and 3.3VA on the other. I would have expected to therefore measure 1.85V on one side of the resistor and 3.3V on the other side of the resistor, but I measure about about 3.4V on both sides. I think that would ordinarily suggest a short on the resistor, but if I measure voltage at directly at pin 1 of the IC PU19 chip itself, the PGOOD pin, it does show as 1.85V, and if that pin 1 is really connected to one side of the PR225 resistor like it shows in the schematic, I would have expected those voltages to be the same, so I'm not sure what is going on there. It almost seems like pin 1 at 1.85V of the IC isn't connected at all to PR225 which reads at ~3.4V on both ends. Maybe I'm just not understanding something fundamental. With the power off I measure resistance from each end of PR225 to ground as 1.86 kOhm and 11.94 kOhm, the difference of which is 10.08 kOhm which is the proper spec for the resistor and tells me the resistor itself isn't shorted.

            PU19 PGOOD at 1.85V doesn't appear at PU21 EN (~0.5V):
            So that PGOOD signal as 1.8VA_PGD from PU19 should be going to PU21 on pin 11, which mysteriously reads ~0.48V. There should be no other input influencing that pin 11 (PJ34 and PJ36 are confirmed open as shown in the diagram). Only other component on that pin 11 diagram path is PC207 which appears on the board as two tiny blobs of solder with one terminal which reads ~0.5V and the other 0V. I see a small scratch near this PC207 component. Checking for other components on the board with a matching description of 0.01u_25V_X7R_04 in the schematic, I see that they are all very tiny light brown capacitors, not just two small blobs of microsolder. So it appears that somehow something tore off PC207 from the board and also made a small scratch near there. About 1 inch away from PC207 is the fan connector for one of the fans I replaced and I do recall that the original component was very difficult to remove. I'm thinking I must have caught PC207 when I pulled the fan connector out and tore it off the board and left a scratch. Oops! That was an expensive and time consuming mistake!

            Does only a missing PC207 cap explain all the problems?
            It looks like I can readily purchase more of those caps (looks like mfg p/n: C0402C103K3RACTU available on ebay, mouser, etc), and I'm willing to learn how to microsolder and to buy whatever tools I need to do so since I find this work interesting... But could this missing capacitor really explain this? Just guessing, but I would think that missing PC207 capacitor would be a noise filter and just on its own could not explain the unexpected voltage drop between 1.8VA_PGD on PU19 at 1.85V, and the input at PU21 pin 11 where is reads ~0.48V, as well as the strange resistor voltage reading of PR225 which is ~3.4V on both sides. If replacing PC207 doesn't fix the problem, maybe PU21 itself is not functioning correctly.

            See attached picture for visual depiction of parts and voltages.

            Plan of action:
            1) Get PC207 component, learn how to micro-solder, microsolder a new PC207 into the correct place
            2) Try to get opinion from board experts on what the strange voltage readings on PR225 may indicate
            3) Repeat troubleshooting if not fixed after replacing PC207

            Thank you very much for participating in my steep learning curve here!

            John
            Attached Files

            Comment


              #7
              Re: System76 partial power-on, VDD1.05 out fail

              A single capacitor rarely causes a big issue, so I don't think so that's the case here. You did a good job to identify PR225, are you sure it's so far away from PU21? However, I don't think it's coincidence when PR225 and the scratch are so near. So we have to assume a broken or cut line somewhere in this area.

              Do you have a microscope by any chance to share better pics? I guess the issue will be found at one of these spots.

              Edit: I won't expect a shorted resistor, since this almost never happen. Usually they will fail open. Also 3.4V from the Pull-up PR225 makes way more sense than only 1.8V what you have measured at Enable pin. I guess you've just confused the pins (check for continuity without power).

              So the actual conclusion is, the rail is fine from PU19 to PR225. The issue should be between PR225 and PU21. Proceed with visual and multimeter inspection there.
              Attached Files
              Last edited by Sephir0th; 09-19-2022, 12:36 AM.
              FairRepair on YouTube

              Comment


                #8
                Re: System76 partial power-on, VDD1.05 out fail

                Thanks @Sephir0th! I don't have a digital microscope yet, but think I'll begin shopping one and will probably have one within a few weeks for better pictures. I work fairly long hours during the week, but hopefully I'll get a chance to follow up on your suggestions tonight. More soon!

                Comment


                  #9
                  Re: System76 partial power-on, VDD1.05 out fail

                  Hi @Sephir0th:

                  You're correct about me confusing the pins on PU19: when measured from proper pin 1 of PU19, voltage is ~3.4V, matching what I see on PR225, so mystery solved there, thank you.

                  PR225 is indeed that far away from PU21 -- see attached image with layout markings highlighted in red.

                  With power off, I did measure from the topside of PC207 (non-ground side of the missing cap solder point) to pin 11 on PU21 which looks like it should be a direct connection, and I get about 1.5 Ohm resistance.

                  With power off, measuring from either side of resistor PR225 to pin 11 on PU21 and I get ~18 Mohm.

                  I did a little more work with the multimeter to verify the above, but with my magnifying glass and the multimeter tool I have, I'm pushing the limit of what I'm able to measure and see. Those components are so small and clustered very close together it's hard to see the lines in the area and where they go.

                  It does look like two separate lines may have been damaged by that scratch, but I need a microscope to confirm that and to be able to clearly see where those lines go to investigate further.

                  John
                  Attached Files

                  Comment


                    #10
                    Re: System76 partial power-on, VDD1.05 out fail

                    Meanwhile after checking the boardview I disagree with PC207 is missing from the board also the pads does look like from factory. The "*" in the beginning of the component description in schematics implies it is not present in this revision of the board. So nothing to worry about anymore. Especially if the 3Volts at Pin11 or the respective Pad of PC207 are present.

                    I'm of good hope we'll find what has failed soon.

                    Greetings
                    FairRepair on YouTube

                    Comment


                      #11
                      Re: System76 partial power-on, VDD1.05 out fail

                      Oh awesome! I didn't know that about the "*", thank you for pointing that out.

                      I ordered a pair of ultra fine multimeter probe tips last night as my current probes are too thick to make it easy to measure the small components and close pins.

                      I've started shopping the microscope. I'm thinking I'll choose a digital microscope with a CCD camera and USB/HDMI interface rather than stereoscopic microscope for PCB. My price range will probably be around $200 - $300 USD (maybe a bit more) which looks like it can probably get me a budget PCB microscope setup that while not necessarily high quality, hopefully will be sufficient for this use case. I'll probably order from Amazon so I can return easily if there is a problem or the magnification is insufficient and I need something better. One of the companies I may try is http://www.hayear.com/ as their PCB microscope products are readily available on amazon for shipping.

                      If you have any recommendations on microscope product, feel free to let me know!

                      Comment


                        #12
                        Re: System76 partial power-on, VDD1.05 out fail

                        Just ordered this CCD microscope kit to try:

                        https://www.amazon.com/HAYEAR-C-Moun.../dp/B08BF8DC6H

                        Should have it by the weekend.

                        More toys! I mean, tools!

                        Comment


                          #13
                          Re: System76 partial power-on, VDD1.05 out fail

                          I've received the microscope and am giving it a try. Really impressed. I can actually see the board clearly now, imagine!

                          I've taken some screenshots of the area both while dry and while wet with 99% isopropyl alcohol (isopropanol). While the board is wet with alcohol the visibility of the traces is increased.

                          In case I need to do some microsoldering once I figure out root cause, I've ordered a JCID AIXUN T3B Intelligent Soldering Station With T210 Series Handle (youtube review https://www.youtube.com/watch?v=Ds2Xq3dwV30), although that will take a few weeks before it comes in. I'll also order a practice microsoldering board so I can get a little experience before I start doing any soldering on a board I actually want to do a good job on.

                          Based on the pictures showing some of those lines being torn open from the scratch, it looks like I could probably also use a little bit of UV mask to cover up the trace lines once the repair is finished.

                          Hopefully a bit later today I'll get some time to do a little more circuit investigation and be able to follow traces better now that I can see detail with the microscope. If not today, next weekend.

                          John
                          Attached Files

                          Comment


                            #14
                            Re: System76 partial power-on, VDD1.05 out fail

                            Woohoo! I think I've got the problem figured out!

                            The scratch has disrupted low resistance electrical continuity between PC207 and PJ34 on the board. PJ34 carries 3.4V on the positive side, and this should be going to PC207 positive side, but only ~0.5V is measured. That voltage on positive PC207 goes directly back to the enable pin of PU21. Measuring resistance between those two components when the board is not powered shows open and it should be a short. See attached.

                            So while the microscope shows the scratch disturbed the path between PC207 and PJ34 and it *looks* like it's still at least partially intact, it doesn't have direct continuity anymore, and it's getting the 0.5V from another path.

                            In any case, to test the theory that the problem is due to disruption of the line connecting positive sides PC207 and PJ34, I manually bridged those two components with an unplugged ultra-fine multimeter probe, pressed the power button, and the white LED came on and stayed on. See attached.

                            I had mentioned this happening above just once and I couldn't figure out why it happened and I couldn't repeat it. I must have been measuring PC207 at the time with my old fat multimeter probes and the probe must have slipped and bridged the two components unintentionally right when I turned it on, so it worked just that once.

                            I checked the other traces which showed damage from the scratch and there appears to still be good electrical continuity on them, so I think this may be the only problem that needs to be fixed.

                            Plan for repair:

                            In a few weeks when the micro-solder iron comes in, I'll plan on repairing the short path between PC207 and PJ34. I'll do some research on the best approach, but perhaps just flowing some additional solder between them along the existing line may be enough. Perhaps some small surface grinding to remove the rest of the green conductor covering along the trace prior to flowing additional solder would be good. If anyone wants to suggest a technique here, please do!

                            After fixing the path between PC207 and PJ34, and verifying the laptop boots again, I'll apply a very small amount of UV mask over the other exposed lines and harden it.

                            When I've got this done, and assuming I've got my laptop back to normal, I'll post a microscope pic of the repair.

                            Happy with the progress! This has been quite a learning journey!
                            Attached Files

                            Comment


                              #15
                              Re: System76 partial power-on, VDD1.05 out fail

                              Very good, I'd even put a wire there to be sure but just a bit of solder might be good enough too and also to put the solder mask there. I knew you'll find it. Well done
                              FairRepair on YouTube

                              Comment


                                #16
                                Re: System76 partial power-on, VDD1.05 out fail

                                Thank you! I'll give the wire a try!

                                Comment


                                  #17
                                  Re: System76 partial power-on, VDD1.05 out fail

                                  Finally got some time to follow up and finish this up!

                                  I'm writing from my now fixed laptop -- all good again! Woo! Amazing what havoc a tiny little scratch that the eye can barely see can do!

                                  Today I practiced microsoldering a small wire jumper on a practice microsoldering board and coated with UV mask, all went well.

                                  Then I moved on to the damaged motherboard itself. The two adjacent components were so close to each other with a partial trace still between them that a sturdy solder bridge just naturally and immediately formed between them when I started to solder and no jumper was necessary. I covered it in UV mask and cured it. I also UV masked and cured the two other areas that had some scratch damage and exposed the traces a bit.

                                  Back up and running now, with the new quiet fans I was initially swapping out when I accidentally scratched the board. The new graphite thermal pads instead of thermal paste seem to be doing well too. Thanks for the help! This was an interesting journey and I got some cool tools out of it -- all for cheaper than buying a replacement board.

                                  John
                                  Attached Files

                                  Comment

                                  Working...
                                  X