Feature to not include daytime images in startrails

[gingerbreadassassin]

As the title suggests, it would be nice to omit daytime images from startrail generation. Timelapse and keograms look great with both included, as you get to see the state of the sky over the entire 24 hours, but startrails (in my opinion) do not benefit.

Example keogram (ignore bird poop I cleaned off halfway through the day):

Example startrails (from previous day; bird poop still present):

Perhaps prefixing each image with day/night in the filename might work, though I haven't dug into the generation scripts to see what the ramifications might be.

[prutsky]

Startrails are generated based on a value in config.sh: BRIGHTNESS_THRESHOLD

Depending on that number (default = 0.1) the images are used in the generation of the startrail images.
Value 0.1 was to dark for me, so I use value 0.25. No daytime images are used while generating the startrail images.

[gingerbreadassassin]

Not sure if it's just my skies, but there seems to be some daytime images, likely close to dawn/dusk, that meet the criteria set in BRIGHTNESS_THRESHOLD.

Here's the same data using 0.15:

0.12:

0.11:

As you can see, the clouds are still present while startrails are only just beginning to form.

[EricClaeys]

@gingerbreadassassin, our "to do" list includes the ability to optionally have a start and stop time for startrails, keograms, and timelapse. It won't be in the next release but hopefully the one later this year. Several people have requested it.

For startrails you really want to use the brightness threshold instead of a set time because a set time may include images that are too bright or eliminate images that are dark enough.

[prutsky]

There is also the overlay variable "DAY_OR_NIGHT" which you can use to check. You shouldn't see DAY in the generated startrail image at the and of the night.
Another variable that is maybe "interfering" with this is the Sun height, default -6 degrees.

[gingerbreadassassin]

Good callout; I just added that variable to the overlay. We'll see tomorrow morning what shows up.

Also, I have my transition set to -10 degrees. This should also make day "come earlier" so to speak, so if anything, this would make it less likely to have daytime images in startrails, if a feature to exclude them existed.

[gingerbreadassassin]

No "Day" images in last night's startrail (variable in top right). I wonder if those clouds would've been considered "Night"

[EricClaeys]

@gingerbreadassassin, Connor, that startrails looks pretty good. What, if anything, did you change?

[prutsky]

Yes, very curious about your settings now:
BRIGHTNESS_THRESHOLD
AUTO_STRETCH
AUTO_STRETCH_AMOUNT
AUTO_STRETCH_MID_POINT

Auto-exposure
Mean Target
Mean Threshold
Auto-Gain

Could you provide these please. You nighttime images are way better than mine.

[gingerbreadassassin]

I'm still tuning the stretching, but current values are

true
20
"8%"

BRIGHTNESS_THRESHOLD=0.12
Auto-exposure=yes
Mean Target=0.15
Mean Threshold=0.01
Auto-Gain=no
gain=1

I used to have autogain enabled and max gain set to 16, but the pi hq camera is so noisy, even just barely over 1, and darks didn't seem to help too much, so I disabled it entirely. I'm working on a TEC cooling system to see if that helps. I'll post about it when I get a little further along.

I'm fortunate enough to live in bortle 2 skies, though I do have some annoying neighbors I need to have a chat with. We're in a dark sky protection area, and they have some lights that are unshaded and way too bright.

Past couple of nights have been completely overcast, else I would post more recent images.

[gingerbreadassassin]

I do yearn for the ability to tune the stretching with a static image instead of waiting for each new sample to come through.

[prutsky]

Thank you for these values. I live under a Bortle class 5 sky, so I probably need the tweak these settings a bit, and compare with the values currently use.

[EricClaeys]

@gingerbreadassassin, Connor, to stretch an image during testing, run this on an UNstretched image:

convert INPUT_IMAGE -sigmoidal-contrast  AMOUNTxMIDPOINT OUTPUT_IMAGE
# e.g.,
convert ~/allsky/images/20240101/image-xxxx.jpg -sigmoidal-contrast 10x20% new_image.jpg

Replace INPUT_IMAGE with the name of your test image that you'll keep the same. Make sure it has NOT been stretched already.
Replace AMOUNT by the stretch amount.
Replace MIDPOINT by the midpoint. (don't forget the x between the numbers and the %.
Replace OUTPUT_IMAGE by some temporary name. I like to use: AMOUNT_MIDPOINT.jpg so I get a new image for every test run and can easily compare them.

[gingerbreadassassin]

ran

for i in {1..20} ; do
   for j in {1..20} ; do
      convert /mnt/nfs/allsky/archive/test-image.png -sigmoidal-contrast ${i}x${j}% /mnt/nfs/allsky/archive/test-image-stretched_${i}x${j}.png 
   done
done

and I think the sweet spot for me is 15x5:

images

no stretch:

15x1

15x2

15x3

15x4

15x5

15x6

15x7

15x8

15x9

15x10

[EricClaeys]

@gingerbreadassassin, Connor, glad you found something that works for you.
Also nice to see you know bash. Most people don't so I suggested the simple way to do it. I used something similar to what you ran when I created the "stretch" documentation page. In the next Allsky version it'll have a for loop as well.

If you consider the original issues as resolved, would you please close this Discussion?

[gingerbreadassassin]

I'm an SRE by day, so I know just enough to be dangerous. Appreciate the edit though; that's much easier to read than my one-liner.

I believe my original issue was not having enough dynamic range in the selection of images, likely from the excess red noise caused by too aggressive of a gain setting. Stumbling across using stretch over gain I think is the right solution. Discussion closed.