First time deploying any sort of app so forgive me if the answer is super obvious.
I’ve got a raspberry pi that’s continuously ingesting sensor information at a rate of 100hz+
I’d like to somehow send the sensor data directly to a dashboard hosted on Qovery without the use of a database.
I’ve never attempted anything like this before and am unsure where to begin. What are the standard, best practices? Through limited research I’ve come across websockets but am unsure if this is the best path to pursue?
Any suggestions, links, or reading material is super appreciated! Preferred language is Python if it matters.
I’ve got a prototype dashboard built using Streamlit. The raspberry pi is monitoring a CAN bus. For each unique id on the bus, the dashboard creates a live plot with the payload data of the last 100 messages. There can be about 15-30 id’s in total at any given time. I’ve got this working successfully locally over TCP at the moment.
Hopefully I interpret this correctly, but the volume is only a single bus connected to the pi.
Each message is a max of 8 bytes.
Definitely a hobby project – just a learning experience and an opportunity for me to get my feet wet!
I’ve done a bit of reading about gRPC per your suggestion and it definitely seems like a great solution. I also came across an alternative RPC, Pyro5, which seems a bit higher level and easier to prototype with.
I look forward to any suggestions! Thank you again for the help.
I took a look and Streamlit looks very cool product. Thanks for sharing.
From what I see, the question is, do you need real-time analytics or not? Because real-time means keeping the connection open with your server app to display the data. So GRPC / Websocket or even your own TCP protocol will fit this purpose. But all of that at the cost of more complexity to stream the data in real-time.
On the other hand, if you don’t need a real-time info display, you can batch the data from your raspberry and send it every X time or X batch size. This is better if your raspberry internet connection is not so good.
It’s just a matter of choice, I don’t see any difficulty here.
gRPC for real-time is a good choice. Approved by several big companies I don’t know about Pyro5. But gRPC is not so difficult but definitely required some time to dig into. Maybe this tutorial fits your need?
I implemented a very basic gRPC example locally on my laptop and got the server successfully working in a container. I’m able to make calls from the client via localhost:50051with no issue.
Dockerfile
FROM python:3.8
EXPOSE 50051
WORKDIR /app
RUN pip3 install grpcio grpcio-tools
COPY . .
CMD ["python", "-u", "server.py"]
I then tried deploying the server on Qovery which was successful. I made the app and port publicly accessible which to my understanding essentially binds that port to the router url(?). But when I try connecting to the url from my client I always get a failure. Is this not possible?
Again, I apologize if this is obvious. My knowledge regarding cloud infrastructure/networking is extremely limited!
Unfortunately at the moment is impossible to route gRPC traffic outside of the private network with Qovery (can you confirm @Pierre_Mavro / @Erebe ?). Better support of other protocols than HTTP will be available in our next release planned for September 2021.
