In my recent discussions with NCSA on gratia, I found something that is probably obvious to other people, but I would still like to point out.
Status captured 6-8-11
According to the OSG PR Display, the OSG runs and accounts for 420,000 jobs daily. That's 4.8 records a second added into MySQL JobUsageRecord table. Also, for each job, summary data is updated in the summary table. So that's at least 5 inserts/updates a second, but most likely many more. All the while, we are able to interactively query the database (well, a replicated one) using pages such as the UNL graphs.
We also keep track of transfers, but they are summarized and aggregated at the probe before reaching the database.
So, in summary, kudos to the gratia operations team.
(NOTE: I may have forgotten some optimizations that we use. But still, that's a lot of records)
The XRootD Monitoring Collector (collector) receives file transfer accounting messages from XRootD servers.
This transfer information is parsed by the collec...
Creating dashboards and data visualizations are a favorite past time of mine. Also, I jump at any chance to learn a new technology. That is why I have spen...
Optimizing data transfers requires tuning many parameters. High latency between the client and a server can decrease data transfer throughput. The Open Scie...
The validation project for XRootD Monitoring is moving to phase 2, scale
testing. Phase 1 focused on correctness of single server monitoring. The
report is...
Leave a comment