Current data, fast answers
What is asked often is held in memory; what grows stale is renewed in the background.
Load once, keep in memory
A search compares more than a hundred spots at once. To do that in milliseconds, the spot list and the precomputed statistics are held in the server’s memory. They are loaded from the database at start-up – and the spot list again as soon as someone saves a spot. Only a spot’s description is read fresh from the database on every call, so a change applies at once.
Serve, then renew in the background
Measurements and forecasts are not fetched for every question. Every request gets the copy in memory at once; if it is too old, the server fetches the new one in the background and the next request has it. Measuring stations are read every five to ten minutes, forecasts after half an hour at the latest. If a source fails, the last valid state stays and is marked as such.
Background jobs
What takes effort is done by jobs: new runs of the ICON-D2 model and of the wave forecast are read as soon as they appear; the five-year statistics are recomputed when they are a month old, a year has closed or a spot was moved – and stored as a result, so a restart has nothing to fetch again.
Demand drives the load
A spot someone asks about is renewed more often than one only the overview map shows. Forecasts are transferred compressed and only when they changed; of large model files only what the spots need is read. The data providers’ limits are respected.
A new spot
A newly created spot is in the list and on the map at once, its description goes into the knowledge base automatically when it is saved, and its statistics are computed within an hour. Until then the Spotfinder does not recommend it but counts it as “not yet with statistics”.
Does this fit your plans?
Tell me in a few sentences what it is about – process, data, goal. We will sort out the rest in a conversation.