The purpose of this DIY project, was to completely automatize the operation of my backyard observatory, which was lacking a sky-condition safety monitor.

Based on the indi-allsky software, I put together a weatherized all-sky camera, using an old QHY5L-II camera I had lying around, a CS-mounted CCTV camera lens (1.6-3.4 mm fisheye, set at 2 mm), a raspberry pi 4B running astroberry/INDI server, and a GoPro acrylic dome (used for underwater shooting).

That took care of the all-sky image acquisition, but was missing an automatic determination of the sky condition. For that I developed an AI-based image classifier, that reads the all-sky camera image stream, and writes to a text file the current sky condition, according to the following 4 classes: Clear, Cloudy, Covered, and Rainy. Together with the class, the app writes a session recommendation, which can be GO, PAUSE or STOP, based on some user defined initial settings and some internal logic. Then, a Safety Monitor in NINA (using ASCOM Generic File SafetyMonitor Driver) reads said text file, and flags the occurrence of certain words as unsafe. Such events are also defined by the user in the safety monitor configuration. This allows for quite a bit of freedom in defining the logic of what to do when.



SkyCondition App:

UPDATE 2024-04-13: At the request of some users, I have modified the app so that input parameters can be loaded from a .csv file instead of dialog windows. This allows for a more autonomus start-up of the process, without the need for human intervention. Such file should be named parameters.csv, and should be placed in the same directory as the app and the keras model file. Its format is a a single line with 4 comma-separated parameters: check period in minutes (integer), Cloudy tolerance in minutes, Covered tolerance in minutes, and the latest skycam image file location (full path with filename included, without quotation marks).
I am including links to the bundled .exe file to run in windows, and the .py file to run in a python environment. Such environment has to include tensorflow 2.12 for the keras model to be compatible, and pillow. (the older app version also requires opencv-python and easygui).

UPDATE 2022-05-11: Minor change to the logic. Now a Covered condition increases both Cloudy and Covered counters.

The app is written in Python, and uses Google´s Tensorflow and Keras, among other libraries. It leverages on Google´s trained MobileNet, a 28-layer CNN, by using transfer learning. In this way the net can be retrained easily using a moderate number of training examples (in this case, about 100 per class). Moreover, this can be done easily online at teachablemachine.withgoogle.com.

The app can work with any all-sky camera serving images to a file location (on a network too). The image stream can be color or monochrome, and of any size (as long as it is 224x224 or larger). The app takes care of data reshaping and normalization internally.

SkyCondition is a bundled, self contained app for Windows 11 (haven´t tried it on other versions), that has no prerequisites or dependances, with the exception of a trained keras model that has to live in the same file location as the app.

To run it, just double-click on it. It will open a dialog window, where the user has to enter, in minutes, the Check Period (how often is the sky condition checked), the Cloudy Tolerance (how many Cloudy uninterrupted minutes are acceptable before issuing a STOP recommendation), and
the Covered Tolerance (how many Covered uninterrupted minutes are acceptable before issuing a STOP recommendation). Note that the occurrence of a Clear condition, resets both Cloudy and Covered counters, while the occurrence of a Cloudy condition resets the Covered counter. A Rainy condition issues a STOP recommendation immediately and freezes both counters. After these parameters are set, the app opens another dialog to point to the all-sky image location. The output text file, named status.txt, is saved at the same location where the app was run from.

You can download the Windows app here, and the keras model that I trained for my location and hardware, here. You can also try run the python code directly, inside the proper python environment (with tensorflow, keras, numpy, pillow, opencv-python, easygui). Not recommended, as dependancies are tricky, but you can try if you know what you are doing. Here is the code.

Check the labeled timelapse videos in the examples.

Some caveats:
1. Note that I trained my system under Bortle 8 skies, so the shared keras model will likely not work well under dark skies. In such environment, on a clear night, the Milky Way could be interpreted as Cloudy :)) It can be easily retrianed, thou.

2. The Rainy condition is detected immediately, likely because of the rain drops on the dome. So if it stops raining, the condition detected will change only when those drops have mostly dried out. The dew heater helps accelerate that process.

3. Some special but infrequent sky conditions have not been included (ie, foggy/hazy). I might add another class in the future, and retrain the CNN, once I gather sufficient training examples.

4. The all-sky image has to be well exposed, with an average ADU around 100/256 (8-bit data).

5. The system was trained with night images, so I wouldn´t trust it during the day. It could be expanded to 24 hours, but it may require duplicating the classes, besides retrianing.