Data Processing Documentation
Overview
Brief description of the data processing objectives and scope. Reminder to adhere to data ownership and usage guidelines.
Data Sources
List and describe data sources used, including links to cloud-optimized sources. Highlight permissions and compliance with data ownership guidelines.
CyVerse Discovery Environment
Instructions for setting up and using the CyVerse Discovery Environment for data processing. Tips for cloud-based data access and processing.
Data Processing Steps
Using GDAL VSI
Guidance on using GDAL VSI (Virtual System Interface) for data access and processing. Example commands or scripts:
gdal_translate /vsicurl/http://example.com/data.tif output.tif
Cloud-Optimized Data
Advantages of using cloud-optimized data formats and processing data without downloading. Instructions for such processes.
Data Storage
Information on storing processed data, with guidelines for choosing between the repository and CyVerse Data Store.
Best Practices
Recommendations for efficient and responsible data processing in the cloud. Tips to ensure data integrity and reproducibility.
Challenges and Troubleshooting
Common challenges in data processing and potential solutions. Resources for troubleshooting in the CyVerse Discovery Environment.
Conclusions
Summary of the data processing phase and its outcomes. Reflect on the methods used.
References
Citations of tools, data sources, and other references used in the data processing phase.