...
Ares is built with Infiniband EDR interconnect and nodes of the following specification:
Partition | Number of nodes | CPU | RAM |
---|
Proportional RAM/CPU | Proportional RAM/GPU | Proportional CPU/ |
---|
GPU | Accelerator | ||||||
---|---|---|---|---|---|---|---|
plgrid | 532 | 48 cores, Intel(R) Xeon(R) Platinum 8268 CPU @ 2.90GHz | 192GB | 3850MB | n/a | n/a | |
plgrid-bigmem | 256 | 48 cores, Intel(R) Xeon(R) Platinum 8268 CPU @ 2.90GHz | 384GB | 7700MB | n/a | n/a | |
plgrid-gpu-v100 | 9 | 32 cores, Intel(R) Xeon(R) Gold 6242 CPU @ 2.80GHz | 384GB |
n/a | 46000M | 4 | 8x Tesla V100-SXM2 |
Job submission
Ares is using Slurm resource manager, jobs should be submitted to the following partitions:
...
Ares uses a new scheme of naming accounts for CPU and GPU computing grants. CPU-only grants are named: grantname-cpu, while GPU accounts use grantname-gpu suffix. Please mind that sbatch -A grantname
won't work on its own. You need to add the -cpu or -gpu suffix! Available computing grants, with respective account names (allocations), can be viewed by using the hpc-grants
command.
Resource allocated on Ares doesn't use normalization, . 1 hour of CPU time equals 1 hour spent on a computing core, similarly in the case of GPUs., similar to GPU time. Resources used to calculate account billing include CPUs, memory, and GPUs. Jobs on CPU partitions are always billed in CPU hours, while jobs on GPU partitions are always billed in GPU hours. If your job uses the default amount of memory per core, or less, then the job is billed simply for the time spent using CPUs. If the job uses more memory for each allocated CPU than the proportional amount (consult the table above), then the job will be billed as it would use more CPUs. The amount billed can be calculated by dividing the memory used by the proportional memory per core. The same principle applies to GPUs, where a GPU has the respective amount of memory per GPU and CPUs per GPU. The cost of running a job can be expressed as a simple algorithm for CPUs:
Code Block |
---|
cost_cpu = cpus_used * job_duration
cost_memory = memory_used/memory_per_cpu * job_duration
final_cost = max(cost_cpu, cost_memory) |
and for GPUs, respectively:
Code Block |
---|
cost_gpu = gpus_used * job_duration
cost_cpu = cpus_used/cpus_per_gpu * job_duration
cost_memory = memory_used/memory_per_gpu * job_duration
final_cost = max(cost_gpu, cost_cpu, cost_memory) |
Storage
Available storage spaces are described in the following table:
...