OpenSearch Transform Job: The Case of the Silent Failure and the Ghost Key
Debugging OpenSearch Transform jobs can feel like searching for a needle in a haystack, especially when the error messages ar...
Feb 13, 2026
// research infrastructure
Derek Weitzel
Distributed Research Infrastructure
Designing and operating production-grade distributed research platforms across national collaborations including OSG, PATh, NRP, and IRIS-HEP.
Focus areas: reliable data movement, scalable throughput computing, and long-lived scientific services engineered for production use.
// current focus
- Federated Kubernetes across research domains
- Reliable high-throughput data movement
- Production-grade distributed research platforms
// systems
Projects
OSG National distributed computing partnership delivering throughput computing and data access to research teams. FabAID National fabric of open data services for data-intensive and AI-driven research. PATh CHTC and OSG partnership expanding access to distributed high-throughput computing and the OSDF. National Research Platform Community-owned Kubernetes platform sharing compute, storage, and networking across institutions. IRIS-HEP Institute advancing software and computing R&D for the HL-LHC era of high energy physics. GP-ENGINE Regional GPU infrastructure expanding advanced computing access across the Great Plains. GP-STOR Tiered object storage for research data, distributed across Great Plains institutions. SciTokens Federated, capability-based authorization for distributed scientific computing.
// expertise
Technologies
Data federation & caching
- XRootD — XRootD distributed data access framework
- OSDF — Open Science Data Federation
- CVMFS — CernVM File System
- Ceph
Throughput computing
- HTCondor — HTCondor high-throughput computing scheduler
- Kubernetes
- Helm
- Docker
Authorization & telemetry
- SciTokens — SciTokens capability-based authorization
- JWT — JSON Web Tokens
- OpenSearch
Languages & runtime
- Go
- Python
- Linux
// writing
Blog
Profiling the XRootD Monitoring Collector
The XRootD Monitoring Collector (collector) receives file transfer accounting messages from XRootD servers.This transfer info...
Jan 31, 2024
Dashboards for Learning Data Visualizations
Creating dashboards and data visualizations are a favorite past time of mine. Also, I jump at any chance to learn a new tech...
Sep 14, 2022
StashCache
StashCache is a framework to distribute data across the Open Science Grid. It is designed to help opportunistic users to tra...
Jun 14, 2017
Running R at HCC
The full presentation.There are many methods to run R applications at HCC. I can break these uses down to: Creating a tradi...
Sep 21, 2016
Hiding all the details: Grid jobs in Docker
Table of Contents For a long time, HTCondor has strived to have the job runtime environment be run and defined by the s...
May 18, 2016