Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix for ppstest timeout #6

Open
fitzy89 opened this issue Aug 19, 2023 · 41 comments
Open

Fix for ppstest timeout #6

fitzy89 opened this issue Aug 19, 2023 · 41 comments

Comments

@fitzy89
Copy link

fitzy89 commented Aug 19, 2023

With a recent update to Raspbian, it's necessary to add arm_64bit=0 to /boot/config.txt and reboot in order to get the pps working

@domschl
Copy link
Owner

domschl commented Aug 19, 2023

I am using PPS with arm_64bit=1 but with Manjaro. So there's probably something else going on? Maybe the access rights for /dev/pps0 changed? Do you have some more information (e.g. dmesg logs?)

@fitzy89
Copy link
Author

fitzy89 commented Aug 19, 2023

I was following your guide using the latest Raspbian on a Pi 4B and at the point of running ppstest I was seeing timeout errors, the same issue as mentioned in this post - raspberrypi/linux#5430 which is what led me to the above solution which seems to work fine for me.

@domschl
Copy link
Owner

domschl commented Aug 20, 2023

Interesting! A kernel bug seems to be the cause: a patch intended for 32bit got wrongly applied to 64bit. Did you try the fix suggested in:

raspberrypi/linux#5430 (comment)

namely:

sudo rpi-update pulls/5478

which pulls a new 64bit kernel patch? (so arm_64bit=1 for the patched kernel).

Edit:

A simple sudo rpi-update should probably do the trick too, my guess is that the fix to the i2c-bcm2835 that was required is now awailable via standard kernel updates.

Note however that kernel updates are not recommended for normal use, since there's always the probability that something else breaks.

@OdessaEyes
Copy link

OdessaEyes commented Aug 21, 2023

For me it didn't work.
Rpi3b, 5.15.76-v7+ #1597 SMP armv7l GNU/Linux, Raspbian GNU/Linux 11 (bullseye)
sudo rpi-update or/and add arm_64bit=0 to /boot/config.txt didn't solve the problem.

If you have any other ideas I'd be happy to try.

Linux rpi3b 6.1.45-v8+ #1671 same situation.

@domschl
Copy link
Owner

domschl commented Aug 21, 2023

@OdessaEyes : You might be encountering a different problem, since you are running 32bit Raspbian.
If your error is not timeouts on ppstest with messages like:

time_pps_fetch() error -1 (Connection timed out)

then it might be best to open a separate bug report describing what you tried?

@Nebulosa-Cat
Copy link

after using sudo rpi-update update to Linux Pi-Server 6.1.61-v8+ #1696 SMP PREEMPT Thu Nov 2 16:44:46 GMT 2023 aarch64
still got time_pps_fetch() error -1 (Connection timed out)
maybe only sudo rpi-update pulls/5478 will work?

@Nebulosa-Cat
Copy link

image
and looks sudo rpi-update pulls/5478 not work

@domschl
Copy link
Owner

domschl commented Dec 21, 2023

The issue seems to be fixed with current (2023-12) bookworm release of Raspberry Pi OS. No more patches are needed. Please re-check, and let me know if there are still issues.

@romm32
Copy link

romm32 commented Jan 29, 2024

Hello. I am running the Raspberry OS, I downloaded the current version (2023-12, 32-bit version) and then ran raspi-update. I am now running an updated version from Jan 22 of 2024. I followed all the instructions with a fresh install of the OS (before starting to run the suggested commands, I ran an apt update then apt upgrade), until I got the connection timeout error in the ppstest. I added the arm64bit line to /boot/config.txt, then rebooted, and nothing happened. I also tried with the raspi-update (as previously mentioned), and a reboot, and it is still not working. I am quite new to Raspberry Pis and don't know if there is something I am doing wrong, or if the issue is a bug in the OS, but I would appreciate any help available.
Thanks in advance

@domschl
Copy link
Owner

domschl commented Jan 30, 2024

@romm32 : Non of the fixes should be necessary anymore. Which model of Raspberry Pi are you using?
Please paste the ppstest command together with it's output into the bug-report.
E.g. for me it looks like:

sudo ppstest /dev/pps0
trying PPS source "/dev/pps0"
found PPS source "/dev/pps0"
ok, found 1 source(s), now start fetching data...
source 0 - assert 1706619523.000001477, sequence: 1233465 - clear  0.000000000, sequence: 0
source 0 - assert 1706619524.000001080, sequence: 1233466 - clear  0.000000000, sequence: 0

I should update the instructions once we figure out what happened with your installation.

@domschl
Copy link
Owner

domschl commented Jan 30, 2024

I just checked: there is still ongoing work on this bug here Raspberry kernel pps issue 5430 .
One thing (if you have one of the newer models) you can try is to use Raspberry OS 64bit?

@romm32
Copy link

romm32 commented Jan 30, 2024

I am using a Raspberry 4 Model B. I was first using it with Ubuntu 22, then switched to Raspberry OS and tried both for 32bit then for 64bit (versions from December 2023, following what was mentioned here). I tried the arm64bit change and it didn't work, nor did a raspi-update. The output was similar to yours, except I was getting the following line right after the "ok, found 1 source(s)...".

time_pps_fetch() error -1 (Connection timed out)

Given I am following this guide for my undergraduate final project, and this was one of many things left on my to-do list, I kept trying with different settings yesterday. I eventually switched to Raspberry OS for 64bit, an archive version from September 2022, and everything worked just fine (no need to try any fixes). I know it is not exactly a solution, but I guess this should be enough for the project I am working on. I was able to get chrony working, but I don't see the PPS as one of the sources. My GPS antenna is kind of bad, and I was working indoors, so I am planning on testing everything when I have a decent amount of GPS satellites available (about 10), and hopefully I will see the PPS as one of the sources.

Thank you for your quick answer, I will continue following these issues just in case someone finds another fix I could try in the future.

@aGGreSSiv
Copy link

Hello,
It's an interesting problem. The NTP server is currently installed in 3 different models, pi3, pi4 and pi5. Pi4 and pi5 have the same operating system. The versions are the same. both are 64bit. (I'm not sure if it is independent of the operating system, the kernal versions are the same) However, while pi3 and pi4 work without any problems, I am experiencing this problem on pi5. I tried every suggestion I found here and online. none of them helped. As I mentioned in my previous questions, I am not very good at Linux. If everything is the same, what's left? If you find a solution or see an update released that will solve this problem, can you write it here?

@Nebulosa-Cat
Copy link

after I'm fully manuly upgrade to PiOS lite Bookworm (follow this guide: https://gist.github.com/jauderho/6b7d42030e264a135450ecc0ba521bd8)

and i delete the bcm2708.pps_gpio_pin=4 at /boot/firmware/cmdline.txt
(the cmdline.txt and config.txt looks move to /boot/firmware at bookworm)

now it work normally

@domschl
Copy link
Owner

domschl commented Mar 6, 2024

Thank you @Nebulosa-Cat for this information! I will update the guide.

@aGGreSSiv
Copy link

it shouldn`t be this hard. @Nebulosa-Cat , i thing ill wait for to get fixed with routine updates..

@ZogKarndon
Copy link

I'm having the same trouble. I have a RPi 3b with the Adafruit Ultimate GPS Hat, running Raspberry Pi OS Lite (32 bit, release 2024-03-15) - Bookworm release

My /boot/firmware/config.txt changes are:

dtoverlay=pi3-disable-bt
dtoverlay=pps-gpio,gpiopin=4
enable_uart=1
init_uart_baud=9600
arm_64bit=0

Still, sudo ppstest /dev/pps0 results in:

trying PPS source "/dev/pps0"
found PPS source "/dev/pps0"
ok, found 1 source(s), now start fetching data...
time_pps_fetch() error -1 (Connection timed out)
time_pps_fetch() error -1 (Connection timed out)
time_pps_fetch() error -1 (Connection timed out)
time_pps_fetch() error -1 (Connection timed out)

Clearly I'm doing something wrong, but I don't know what.

@Nebulosa-Cat
Copy link

@ZogKarndon
some of my /boot/fireware/config.txt:

[all]
auto_initramfs=1
enable_uart=1
dtoverlay=pps-gpio,gpiopin=4
hdmi_enable_4kp60=1

and my /boot/fireware/cmdline.txt:

console=tty1 root=PARTUUID=50cc6c23-02 rootfstype=ext4 fsck.repair=yes rootwait  cgroup_enable=memory cgroup_memory=1

and if you not get 3D-Lock yet you will recive time_pps_fetch() error -1 (Connection timed out) too, might be check cgps

@domschl
Copy link
Owner

domschl commented Apr 9, 2024

Some additional checkpoints:

Are you loading both required kernel modules (e.g. in /etc/modules-load.d/raspberrypi.conf):

pps-gpio
pps-ldisc

Please try:

 sudo dmesg | grep pps

We can then compare the behavior of the kernel modules with this:

[    0.000000] Kernel command line: reboot=w coherent_pool=1M 8250.nr_uarts=1 pci=pcie_bus_safe snd_bcm2835.enable_compat_alsa=0 snd_bcm2835.enable_hdmi=1  smsc95xx.macaddr=D8:3A:DD:B1:D6:7A vc_mem.mem_base=0x3fc00000 vc_mem.mem_size=0x40000000  console=serial1,115200 console=tty1 root=PARTUUID=29c1963e-02 rootfstype=ext4 fsck.repair=yes rootwait quiet splash plymouth.ignore-serial-consoles bcm2708.pps_gpio_pin=4 cfg80211.ieee80211_regdom=DE
[    0.031476] pps_core: LinuxPPS API ver. 1 registered
[    0.031478] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <[email protected]>
[    3.155228] pps pps0: new PPS source [email protected]
[    3.155256] pps pps0: Registered IRQ 185 as PPS source
[    3.156639] pps_ldisc: PPS line discipline registered
[    5.551950] pps pps1: new PPS source ptp0
[    7.623051] pps pps2: new PPS source acm0
[    7.623065] pps pps2: source "/dev/ttyACM0" added

The the pps device:

ls -la /dev/pps*

Notice owner of /dev/pps0:

crw-rw---- root _chrony 0 B Mon Apr  8 15:27:07 2024  /dev/pps0
crw------- root root    0 B Mon Apr  8 15:27:08 2024  /dev/pps1
crw------- root root    0 B Mon Apr  8 15:27:10 2024  /dev/pps2

This was achieved via: /etc/udev/rules.d/pps-sources.rules containing:

KERNEL=="pps0", OWNER="root", GROUP="_chrony", MODE="0660"

@ZogKarndon
Copy link

@domschl : I have loaded both kernel modules, and added the /etc/udev/rules.d/pps_sources.rules;

pi@pitimer:~ $ ls -al /dev/pps*
crw-rw---- 1 root _chrony 250, 0 Apr  9 10:17 /dev/pps0
pi@pitimer:~ $ sudo dmesg | grep pps
[    0.100991] pps_core: LinuxPPS API ver. 1 registered
[    0.101018] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <[email protected]>
[    8.073330] pps pps0: new PPS source [email protected]
[    8.073424] pps pps0: Registered IRQ 200 as PPS source
[    8.084709] pps_ldisc: PPS line discipline registered

One thing I'm not clear about: Does the pps signal not appear if the GPS does not have a lock? My Adafruit GPS doesn't have a lock, even with the external antenna (although I'm still trying to figure out if the GPS thinks the antenna is connected).

@domschl
Copy link
Owner

domschl commented Apr 10, 2024

You won't get PPS, until GPS has 3D lock with at least 3-4 satellites. That explains your timeouts. Before even trying to test PPS, you must first make sure that you have stable GPS reception and 'Lock'.

For that you need an active GPS antenna (it should say something like 3-5V on the backside of the GPS receiver), passive antennas look similar but have way worse reception.

The GPS antenna must have unobstructed line-of-sight to a good portion of the sky, the GPS signal doesn't penetrate walls. It's best to fix the GPS antenna to a window, or even try outside, if that's a possibility.

Adafruit do a very good job documenting their hardware, have a look at their description of the test-procedure:

Adafruit GPS hat basic test

@ZogKarndon
Copy link

I moved the active GPS antenna to a better location, and it's working now. (I still haven't figured out how to get the antenna status from the GPS, though; the listed instructions don't appear to work with minicom.)

Now all I have to do is figure out a better mounting strategy for the Pi and its extremely fragile GPS antenna connector.

@aGGreSSiv
Copy link

aGGreSSiv commented Apr 29, 2024

Hello

I want to use the GT-U7 GPS module, which I have used before on pi3 and pi4 without any problems, now with pi5. You know, there was a pps problem. With the updates, the pps problem was solved without me doing anything additional. But as you can see below, I cannot run GPSmon. I changed the bandwidth, I replaced the entire GPS module, but I still couldn't get it to work. When I connect the GPS module to the computer, I see GPS data coming. Has it happened to you? Is there something I missed? (pi5 has the latest updates for both operating system and firmware)

image

In the new pi5 operating system, the directory and file name below is no longer valid( to add pps-gpio and pps-ldisc) ? What if we don't create a new file?
/etc/modules-load.d/raspberrypi.conf

so I added modules here;

image

@domschl
Copy link
Owner

domschl commented Apr 29, 2024

The problem seems to be already with GPS receive not working, and that needs to be fixed before even trying PPS.

In Theory, there should not be any difference between connecting the GPS module to either Pi 4 or 5.

How did you physically connect the the GT-U7 module to the Pi 5? The module seems to be a serial module, so which serial port did you use on the Raspberry? Did you try a cat /dev/<your-port>?

The first step should be to verify that you are receiving the proper serial data. You can always compare that with your other Raspberries.

@aGGreSSiv
Copy link

The problem seems to be already with GPS receive not working, and that needs to be fixed before even trying PPS.

In Theory, there should not be any difference between connecting the GPS module to either Pi 4 or 5.

How did you physically connect the the GT-U7 module to the Pi 5? The module seems to be a serial module, so which serial port did you use on the Raspberry? Did you try a cat /dev/<your-port>?

The first step should be to verify that you are receiving the proper serial data. You can always compare that with your other Raspberries.

The cat /dev/ttyAMA10 command does not give any output.

image

Yes, I think it's GPS related, but I connected it the same way I connected the others. gps tx --> gpio15 and gps rx --> gpio14 and pps --> gpio18. I tried it with two different GPS modules. I didn't need to make any adjustments to the ones I bought before. I will connect it to the computer and check again.

IMG_20240429_173330

@domschl
Copy link
Owner

domschl commented Apr 29, 2024

Did you remember to enable the serial port in raspi-config?

sudo raspi-config

Go to 'Interface Options', 'Serial Port':

You need to switch off 'login shell', and enable the Serial Port.

/dev/ttyAMA10 is an interactive tty (used to login or debug), and can't be used as serial port.

@aGGreSSiv
Copy link

aGGreSSiv commented Apr 29, 2024

I've been trying to solve this issue for about 1 month. I suspected this when I didn't see ttyAMA10 in any documentation regarding GPS. I checked it with the command you mentioned 4-5 times, it looks as it should. Now I did it again anyway. When I check it still shows ttyAMA10.

(By the way, I don't know if it's relevant, but as far as I understand from the documents I read, I set Bluetooth to be disabled at boot in /etc/firmware/config.txt.)

image

after restart;
image

@aGGreSSiv
Copy link

I think this bug is related to my problem

@domschl
Copy link
Owner

domschl commented Apr 29, 2024

You are right! Things have changed with Raspberry Pi 5. That serial0/AMA10 port is now by default the new "Debug" connector that exists only on the Pi 5. You could connect your GPS to that new connector?

Alternatively you can use the new config

 uart0_console           Move the kernel boot console to UART0 on pins
                                6, 8 and 10 of the 40-way header (2712 only,
                                default "off")

I'm not quite sure how that works?!

This video shows some information:
https://www.youtube.com/watch?v=27p4XHE3iyw

@aGGreSSiv
Copy link

aGGreSSiv commented Apr 29, 2024

I watched this video. But when I watched it from my current perspective, I realized that I needed to change my google search.

dtparam=uart0=on"

As a result of my searches, only the above command was found in /boot/firmware/config.txt. When you delete this and write it as follows, it says ttyAMA0 instead of ttyAMA10 in front of the serial port. GPSmon works now.

dtoverlay=
dtparam=uart0
dtparam=uart0_console

image

Thank you for guiding me on the right perspective or what I should focus on.

@domschl
Copy link
Owner

domschl commented Apr 30, 2024

I've updated the documentation, thanks for your research @aGGreSSiv !

@rockrabbit
Copy link

Pi5

using: https://www.adafruit.com/product/4279

I can get a 3D lock

but no PPS

[ 5.158490] pps pps2: new PPS source usbserial0
[ 5.158501] pps pps2: source "/dev/ttyUSB0" added

Any ideas?

@domschl
Copy link
Owner

domschl commented Jun 7, 2024

Please provide the complete output of sudo dmesg | grep pps. Which Raspberry Pi pin did you connect the output of Adafruit's PPS pin to?

@rockrabbit
Copy link

pi@pi250:~ $ sudo dmesg | grep pps
[ 0.037002] pps_core: LinuxPPS API ver. 1 registered
[ 0.037006] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [email protected]
[ 1.774033] pps pps0: new PPS source [email protected]
[ 1.774065] pps pps0: Registered IRQ 173 as PPS source
[ 1.775395] pps_ldisc: PPS line discipline registered
[ 3.201279] pps pps1: new PPS source ptp0
[ 5.134434] pps pps2: new PPS source usbserial0
[ 5.134446] pps pps2: source "/dev/ttyUSB0" added

@rockrabbit
Copy link

This is on the latest Pi5.. connected via USB

@rockrabbit
Copy link

pi@pi250:~ $ sudo ppstest /dev/pps0
trying PPS source "/dev/pps0"
found PPS source "/dev/pps0"
ok, found 1 source(s), now start fetching data...
time_pps_fetch() error -1 (Connection timed out)
^Z
[1]+ Stopped sudo ppstest /dev/pps0
pi@pi250:~ $ sudo ppstest /dev/pps1
trying PPS source "/dev/pps1"
found PPS source "/dev/pps1"
ok, found 1 source(s), now start fetching data...
time_pps_fetch() error -1 (Connection timed out)
^Z
[2]+ Stopped sudo ppstest /dev/pps1
pi@pi250:~ $ sudo ppstest /dev/pps2
trying PPS source "/dev/pps2"
found PPS source "/dev/pps2"
ok, found 1 source(s), now start fetching data...
time_pps_fetch() error -1 (Connection timed out)
^Z
[3]+ Stopped sudo ppstest /dev/pps2

@domschl
Copy link
Owner

domschl commented Jun 7, 2024

Just to make sure:

You know that in addition to the USB connection, you need to solder a cable to Adafruit's PPS output pin and connect that to Raspberry GPIO Pin 4?

(Sorry, I just want to make sure that connection is ok)

@Reinitialized
Copy link

Hello!

I am too running into this issue on the Raspberry Pi 4 8GB model. I intend to perform separate tests on a 5 8GB and 3B+ 1GB if the former also fails.

NOTE: I know I had this working roughly around 2/28/2022 as this is when I purchased the active antenna. To me, something has changed in the software, as I have replicated the EXACT hardware setup I had before.

Hardware:

  • Raspberry Pi Model B 8GB
  • Adafruit Ultimate GPS Hat
  • GPS Antenna

Software:

  • For diagnostics and testing purposes, I am running everything as root to ensure there are no permission issues. I will implement proper security isolation once I know it is at least working for root.
  • I have successfully gotten up to confirming GPSD is working as intended with a confirmed 3D DGPS FIX
    image
  • I have added the documented /boot/firmware/config.txt lines, with some extra ones I found through attempting to fix this issue:
    image
  • I did try arm_64bit=0, but this made my OS unbootable. Currently using default arm_64bit=1.
  • I have added the required kernel modules and confirmed they are loading as expected:
    image
  • However, ppstest continues to fail with a timeout:
    image

Thus, I am at a complete loss.

@Reinitialized
Copy link

Reinitialized commented Aug 28, 2024

So I am not quite sure what happened, debugging this has left me quite tired, but after scouring through various issues on the matter, I found the following command over at raspberrypi/linux#5430 which seems to have kicked the pps signal into gear:
while true; do gpioset gpiochip0 17=1; gpioset gpiochip0 17=0; sleep 1; done & ppstest /dev/pps0

This "fix" has persisted even after a reboot to ensure the loop wasn't still feeding into the signal. I'll try to do some additional digging into exactly what made this work, but throwing it here in case it can save others some much frustrating debugging time on the matter.

@owensresearch
Copy link

Is this still an issue for anyone? I am running a Raspberry Pi 3B+ with the Adafruit Ultimate GPS v3 Serial module. 64-bit OS.

I have tried the fudged PPS trick by putting GPIO 17 & 18 together and running a loop. This works for me.. actually gives me a PPS reading on gpsmon.

I have a 3D DGPS fix.
image

During intial setup/testing i was using a random old cable i had laying around and didn't realize that 2 of the pins were actually connected. So, this resulted in 5V going towards the PPS pin for about 1 hour. Could this have fried my PPS function on the Adafruit? I put a multimeter on the pin and can see 0.01V going up and down by .01 ish about every second. Assume the multimeter can't read the voltage fluctuations fast enough. Tried switching to mA and i can see spikes every second. Seems like i have a pulse.. just not sure if strong enough.

I'm no expert here.. i'm a network guy. Does this sound like my Adafruit module is fried or is this some software issue?

@owensresearch
Copy link

Hello!

I am too running into this issue on the Raspberry Pi 4 8GB model. I intend to perform separate tests on a 5 8GB and 3B+ 1GB if the former also fails.

NOTE: I know I had this working roughly around 2/28/2022 as this is when I purchased the active antenna. To me, something has changed in the software, as I have replicated the EXACT hardware setup I had before.

Hardware:

  • Raspberry Pi Model B 8GB
  • Adafruit Ultimate GPS Hat
  • GPS Antenna

Software:

  • For diagnostics and testing purposes, I am running everything as root to ensure there are no permission issues. I will implement proper security isolation once I know it is at least working for root.
  • I have successfully gotten up to confirming GPSD is working as intended with a confirmed 3D DGPS FIX
    image
  • I have added the documented /boot/firmware/config.txt lines, with some extra ones I found through attempting to fix this issue:
    image
  • I did try arm_64bit=0, but this made my OS unbootable. Currently using default arm_64bit=1.
  • I have added the required kernel modules and confirmed they are loading as expected:
    image
  • However, ppstest continues to fail with a timeout:
    image

Thus, I am at a complete loss.

Any luck? I'm about at the same place as you. Different model Pi.

Have you put any kind of multimeter on the Adafruit PPS pin? I did and it seems like i can't read the 3.3V fast enough.. but i do see a pulse of some sort with mA mode.

Would be curious to hear any progress you made.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

10 participants