Draft:XSDK: Extreme-scale Scientific Software Development Kit

From Wikipedia, the free encyclopedia
  • Comment: Resubmitting without taking into account the comments does not help; there are no substantive differences between this version and prior ones. It needs to be rewritten, focusing upon facts (not claims), all of which need to be documented. For instance, include details of all partners with (independent) sources to demonstrate their involvement. You have to demonstrate that others consider this initiative noteworthy by their words and actions. Ldm1954 (talk) 03:23, 7 April 2024 (UTC)
  • Comment: None of the independent sources appear to actually discuss this topic. Stuartyeates (talk) 03:39, 6 December 2023 (UTC)
  • Comment: See WP:COI. References need to verify statements made, for example "The xSDK's testing is currently conducted using Gitlab CI" cannot just use GitLab documentation, with no mention of the subject, as a source. Greenman (talk) 11:07, 29 November 2023 (UTC)
  • Comment: Use "straight quotes" not “curly quotes” and logical quotes. Use automatic Wiki referencing, not hard-coded [1] [2] [3]. —Anomalocaris (talk) 02:03, 20 June 2023 (UTC)

The xSDK, or Extreme-Scale Scientific Software Development Kit, is a collection of software components, tools, and policies. It facilitates the development of scientific packages by integrating interconnected software components from various independent groups in the high-performance computing community.[1] To qualify as an xSDK member, a software package must adhere to community policies and demonstrate interoperability with other xSDK libraries.[2] The initial release in April 2016 demonstrated the importance of community guidelines in enabling the use and transferability of independently developed software packages. The most recent xSDK release includes 26 mathematical libraries, such as PETSc,[3] Trilinos,[4] and SuperLU,[5] forming a foundation for addressing software interoperability and performance portability challenges.[6] The xSDK aims to achieve a framework that promotes collaboration among diverse teams in the high-performance computing community. By providing interconnected and mutually supportive software components, the xSDK may serve as a building block for application development.[7] Overall, the xSDK aims to establish a software ecosystem that can be installed on popular computing platforms. Its goal is also to initiate a scalable scientific software ecosystem.[6]

Key features[edit]

The xSDK, a scientific computing initiative, possesses two features. Firstly, it adopts a peer-to-peer interoperability approach, utilizing libraries to access functionalities across different packages, which facilitates the integration of packages while preserving the existing code structures. Secondly, the xSDK implements software policies that foster compatibility and complementarity among independently developed packages. These policies establish expectations for design, implementation, documentation, support, and installation, allowing separate package development while ensuring their combination and effectiveness. XSDK libraries also possess capabilities to interact with and modify preexisting data objects that were independently constructed using their own creation procedures.[7]

Interoperability[edit]

The xSDK's interoperability offers a software suite and associated policies focusing on versatility, reliability, and cost-effectiveness.[8] At the core of this interoperability is the utilization of Spack,[9] an adaptable package manager that facilitates support for multiple versions, configurations, compilers, and platforms. By leveraging Spack, the xSDK automates the determination of dependencies between packages using scripts provided by library developers, resulting in a building process for complex application codes.[2] In addition, the xSDK provides level 1 interoperability across all xSDK libraries by providing a build script, testing environment, and a set of community policies, guaranteeing collaboration and compatibility for users.[10]

Testing[edit]

The xSDK's testing is currently conducted using Gitlab CI,[11] ensuring testing across various platforms. Efforts are underway to enhance the testing capabilities by introducing new tests and additional layers of testing. One such improvement involves testing a subset of xSDK packages to facilitate the identification of potential issues. The main purpose of CI testing is to ensure the stability of the current release version of the xSDK. However, an important secondary benefit is the ability to test with development versions of different libraries, including those with CUDA[12] or OpenMP[13] capabilities, particularly on systems equipped with GPUs[14] or multicore nodes.[2]

Community policies[edit]

The xSDK, upon its initial release, introduced a set of community policies aimed at enhancing the usability, adaptability, and durability of software packages. These policies specifically tackled the hurdles associated with software interoperability and performance portability. Being xSDK compatible results in integration with other packages, facilitating a cohesive software environment. Moreover, the community installation policies established by xSDK assist in the configuration and installation procedures across diverse platforms. Through the implementation of these policies, xSDK fosters long-term sustainability and promotes interoperability among packages, forming a foundation for complex applications.[7]

Scientific applications[edit]

The xSDK's scientific applications encompass diverse research areas, including surface and subsurface hydrology. Scientists at the Department of Energy utilize software tools like Amanzi[15]]/ATS,[16] CrunchFlow,[17] ParFlow,[18] and PFLOTRAN,[19] along with xSDK numerical libraries such as hypre,[20] PETSc, SuperLU, and Trilinos. They also incorporate Alquimia,[21] a specialized xSDK package, to enhance biogeochemical functionality. These efforts contribute to the development of models for hydrological processes. One application of the xSDK is forecasting the impact of climate change on the Upper Colorado River System. Biogeochemistry models driven by genomic data are integrated with high-resolution hydrology models to understand carbon and nutrient cycles. Stream meanders in the floodplain and the exchange of water between streams and groundwater (hyporheic exchange) significantly affect these cycles. The interconnectedness of groundwater, land surface, and atmospheric feedback is also considered, leading to ongoing studies on ultra-high-resolution integrated hydrology models on regional and continental scales. These models are linked to terrestrial process models and atmospheric models to capture the complexity of the hydrological system.[6]

Future initiatives[edit]

The xSDK project has been expanding its compatibility to include all ECP[22]-funded library efforts, such as SUNDIALS[23]] and Magma,[24] while addressing performance challenges in exascale computing platforms. Efforts to refine community policies and develop domain component collections tailored to specific communities are underway, enhancing interoperability among scientific libraries. Additionally, xSDK is exploring the use of community installation tools like Spack to ensure reliable installation of its libraries. The project aims to develop interfaces for transfer of computational resources between applications and libraries, optimizing performance in top-level MPI processes.[6] In the future, xSDK plans to adapt its community policies to meet community needs and include appropriate packages into its framework. Testing will ensure reliable build and execution, while additional interoperability layers and new example codes will promote collaboration and interoperability between libraries. Specialized xSDK builds with features like CUDA or OpenMP capabilities will be developed to cater to systems equipped with GPUs or multicore nodes. These initiatives will foster collaboration within the scientific computing community.[2]

See also[edit]

Exascale Computing Project (ECP)

PETSc

SUNDIALS

Magma

Spack

CUDA

OpenMP

Scientific computing

Interoperability

High-performance computing

References[edit]

  1. ^ Heroux, Michael (22 May 2023). "Scientific Software Ecosystems" (PDF). CSBSJU.
  2. ^ a b c d Yang, U.M.; Balos, C.; Luszczek, P.; Osborn, S.; Willenbring, J. (22 May 2023). "Challenges of and Opportunities for a Large Diverse Software Team" (PDF). GitHub.
  3. ^ "PETSc 3.20 — PETSc 3.20.1 documentation". petsc.org. Retrieved 2023-11-29.
  4. ^ "Trilinos Home Page | Trilinos". trilinos.github.io. Retrieved 2023-11-29.
  5. ^ "SuperLU: Home Page". portal.nersc.gov. Retrieved 2023-11-29.
  6. ^ a b c d "xSDK". Retrieved 2023-11-29.
  7. ^ a b c Bartlett, Roscoe; Demeshko, Irina; Gamblin, Todd; Hammond, Glenn; Heroux, Michael; Johnson, Jeffrey; Klinvex, Alicia; Li, Xiaoye; McInnes, Lois Curfman (2017-02-27). "xSDK Foundations: Toward an Extreme-scale Scientific Software Development Kit". arXiv:1702.08425v1 [cs.MS].
  8. ^ "The Exascale Software Portfolio". str.llnl.gov. Retrieved 2023-11-29.
  9. ^ "Spack". Spack. Retrieved 2023-11-29.
  10. ^ "What are Interoperable Software Libraries? Introducing the xSDK". bssw.io. Retrieved 2023-11-29.
  11. ^ "Get started with GitLab CI/CD | GitLab". docs.gitlab.com. Retrieved 2023-11-29.
  12. ^ "CUDA Zone - Library of Resources". NVIDIA Developer. Retrieved 2023-11-29.
  13. ^ tim.lewis. "Home". OpenMP. Retrieved 2023-11-29.
  14. ^ "Graphics Processor Unit (GPU) | What, Type, How & Definition". 2020-06-02. Retrieved 2023-11-29.
  15. ^ Laboratory, Los Alamos National. "Earth and Environmental Sciences | Organizations". Los Alamos National Laboratory. Retrieved 2023-11-29.
  16. ^ Xu, Zexuan; Molins, Sergi; Dwivedi, Dipankar; Özgen-Xian, Ilhan; Steefel, Carl (2022-01-01). ATS (Advanced Terrestrial Simulator) integrated hydrology and reactive transport model output in Copper Creek, Colorado (Report). Environmental System Science Data Infrastructure for a Virtual Ecosystem (ESS-DIVE) (United States). OSTI 1877377.
  17. ^ "0.4 CrunchFlow | PNG 550: Reactive Transport in the Subsurface". www.e-education.psu.edu. Retrieved 2023-11-29.
  18. ^ GOERGEN, Stefan KOLLET, Reed MAXWELL, Laura CONDON, Klaus. "ParFlow hydrologic model". parflow.org. Retrieved 2023-11-29.{{cite web}}: CS1 maint: multiple names: authors list (link)
  19. ^ "PFLOTRAN". www.pflotran.org. Retrieved 2023-11-29.
  20. ^ "HYPRE | Computing". computing.llnl.gov. Retrieved 2023-11-29.
  21. ^ Andre, Benjamin; Molins, Sergi; Johnson, Jeffrey; Steefel, Carl (2013-08-01). Alquimia (Report). Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States). OSTI 1231758.
  22. ^ "Home Page". Exascale Computing Project. Retrieved 2023-11-29.
  23. ^ "SUNDIALS | Computing". computing.llnl.gov. Retrieved 2023-11-29.
  24. ^ "Magma Computational Algebra System". Magma. 2 June 2023.
Retrieved from "https://en.wikipedia.org/w/index.php?title=Draft:XSDK:_Extreme-scale_Scientific_Software_Development_Kit&oldid=1218338087"