Thread 'Recommendation about running 2 GPU tasks in parallel?'
Message boards : General discussion : Recommendation about running 2 GPU tasks in parallel?
Message board moderation
| Author | Message |
|---|---|
|
New member Send message Joined: 29 Oct 26 Posts: 3 Credit: 2,265,119 RAC: 67,661  
|
Hello, it seems that after ~50% progress of the GPU tasks the GPU is going to idle and only the 2-3 CPU cores are finishing the task. Would it make sense to start a second task in parallel or will the tasks get in each other's way? |
|
Send message Joined: 23 Sep 26 Posts: 228 Credit: 44,185,220 RAC: 1,478,559  |
In reply to Magiceye04's message of 31 Oct 2026: Hello, Hi Magiceye04, good question, and what you see is real: towards the end of a task the work moves to the CPU and the GPU sits mostly idle. That's why a second task can make sense. Running two tasks per card isn't officially validated yet, so it's your choice. But I checked the last 48 hours: about 35 machines clearly running more than one task per card failed no more often than the others (about 1.7% versus 1.6 to 2%). Each task gets slower, but more tasks finish per day. As an example, one RTX 4070 doing this completes about 23 tasks a day, while a single task at a time at its speed would give about 15. That's one machine, not a controlled test. If you'd like to try it on your RTX 4070, put this in an app_config.xml file in the project folder, then use Options → Read config files: <app_config> <app> <name>eternity_gpu_272_32768v</name> <gpu_versions> <gpu_usage>0.5</gpu_usage> <cpu_usage>2</cpu_usage> </gpu_versions> </app> </app_config> Two is the most that makes sense, as the project sends at most 2 GPU tasks per card. Please tell us how it goes! |
|
New member Send message Joined: 29 Oct 26 Posts: 3 Credit: 2,265,119 RAC: 67,661
|
Thanks Benjamin, it is running now this way. The reason of my question is also that I found actually one task that seems to run without further progress. Not only 2 or 3 hours like other tasks, but now nearly 6hours. It does not block the GPU any more with a second task running in parallel. I think I can cancel this one, as the stderr.txt was not updated since 2,5 hours (around the time when it switches from GPU to CPU part):
2026-10-31 08:45:25 (831277): wrapper (8.3.26019): starting
2026-10-31 08:45:25 (831277): wrapper: running ../../projects/boinc.eternityathome.org_eternity/bw_runner_nvidia_common_0.17_linux (--force-interleave --shard-index 385 --shard-count 4096 --force-jitter-start 1869959296 --force-jitters 1870232704 --threads 4 --gpu-slots 0 --node-cap 2000000000 --seconds 0 --exact-endgame 244:50000000 --endgame-probe 4 --endgame-ladder 4:236:1000000000,3:230:6000000000,2:224:64000000000,1:216:80000000000 --ring-breaks 236 --sigma-tail --no-sigma-main --ncon2-direct 1 --mrv-dem-inc 1 --main-fast 1 --min-save 463 --breaks 192,197,201,206,211,216,221,225,229,233,237,241,246,250 --root-allow exploration_allow.txt --seed 6152026 --epoch cloudflare-boinc-explore --tourney tour --ticket-cpu-seconds 0 --runner-tail-fraction 0.005 --runner-idle-handoff 90 --runner-underfill-handoff 90)
2026-10-31 08:45:25 (831277): wrapper: created child process 831279
NVIDIA beta: assigned device 0, automatic single resident wave, current-free-memory budget
pipeline: selected scientific engine e2_bw_nvidia_avx2_0.4
bw_runner: progress 0/32841 (0%), restored runtime 0 s
NVIDIA_RESOURCE_ADMISSION block=640 exit=0
NVIDIA_BETA_PRIORITY inherited_nice=19
[2026-10-31 08:45:26] heuristic colors: {13,16,10}, total edges 122 (ramp scale x1.000)
[2026-10-31 08:45:26] clue mode: FULL-CLUES (5 hints) -> corner-first fill order, heurScale 0.970 (mode default)
[2026-10-31 08:45:26] breaks (14 total, target completion score 466): 192,197,201,206,211,216,221,225,229,233,237,241,246,250
[2026-10-31 08:45:26] P13 SHARD 385/4096 jitterStart=1869959296 : ce worker traite les tickets globaux J0*nroots + 385 + 4096*local (sous-ensemble disjoint)
[2026-10-31 08:45:26] bw starting: 1 threads, seed 6152026, nodeCap 2.0B, heurScale 0.970, minSave 463, resume best 0, seconds 0
[2026-10-31 08:45:26] clue: piece 255 td(2,2) rot 2 -> bu cell (13,2) fill idx 188
[2026-10-31 08:45:26] clue: piece 249 td(2,13) rot 3 -> bu cell (13,13) fill idx 247
[2026-10-31 08:45:26] clue: piece 139 td(8,8) rot 1 -> bu cell (7,8) fill idx 120
[2026-10-31 08:45:26] clue: piece 208 td(13,2) rot 2 -> bu cell (2,2) fill idx 8
[2026-10-31 08:45:26] clue: piece 181 td(13,13) rot 2 -> bu cell (2,13) fill idx 45
[2026-10-31 08:45:26] FORCE_ROOT chargé: 3084 racines, 1870232704 jitters/racine, 1 threads
[2026-10-31 08:45:26] ROOT_ALLOW : 123 cidx autorisés / 3084 racines (32841 lignes lues, 0 hors-bornes ignorées) -> les autres cidx sont sautés (ticket/seed d'origine préservés)
[2026-10-31 08:45:26] TICKET_LOG ARME run_id=ced1b12abc9d1c1e cfg_hash=05810c67dbc7af82 dir=startup_probe/640/ticketlog (segment immuable v2)
[2026-10-31 08:45:26] HARVEST463 dir=startup_probe/640/harvest (board vérifié score>=463, identité SHA256)
[2026-10-31 08:45:26] table sizes: corners 16 leftSides 56 bottomSides 56 topSides 1288 rightNoBrk 56 rightBrk 1288 midNoBrk 764 midBrk 34380
[2026-10-31 08:45:26] ring-break tables (from fill depth 236, 8 border cells): corners 720 leftSides 2520 bottomSides 2520 topSides 2520 rightBrk 2520
[2026-10-31 08:45:26] special[0] (cell of clue 255): 1 entries
[2026-10-31 08:45:26] special[1] (west-nb of clue 255): 45 entries
[2026-10-31 08:45:26] special[2] (south-nb of clue 255): 44 entries
[2026-10-31 08:45:26] special[3] (cell of clue 249): 1 entries
[2026-10-31 08:45:26] special[4] (west-nb of clue 249): 2025 entries
[2026-10-31 08:45:26] special[5] (south-nb of clue 249): 2115 entries
[2026-10-31 08:45:26] special[6] (cell of clue 139): 1 entries
[2026-10-31 08:45:26] special[7] (west-nb of clue 139): 44 entries
[2026-10-31 08:45:26] special[8] (south-nb of clue 139): 45 entries
[2026-10-31 08:45:26] special[9] (cell of clue 208): 1 entries
[2026-10-31 08:45:26] special[10] (west-nb of clue 208): 45 entries
[2026-10-31 08:45:26] special[11] (south-nb of clue 208): 46 entries
[2026-10-31 08:45:26] special[12] (cell of clue 181): 1 entries
[2026-10-31 08:45:26] special[13] (west-nb of clue 181): 44 entries
[2026-10-31 08:45:26] special[14] (south-nb of clue 181): 45 entries
[2026-10-31 08:45:26] GPU: tables in left-major slice order: 3,7,11,17 (the others bottom-major)
[2026-10-31 08:45:26] GPU: table 6 (23 deep cells): 234 of 267 slices' no-break groups described
[2026-10-31 08:45:26] GPU: table 7 (41 deep cells): 234 of 493 slices' no-break groups described
bwgpu: frontier of 22 entries (6 groups)
bwgpu: offsets overlay of 4 words (room 228): depths 45 120 188; 98416 bytes of shared memory a block
bwgpu: break test off (no candidate of a scan's range can have too many breaks)
bwgpu: deep region from depth 192; not described (the other region's tables): 196
NVIDIA_BETA slots=640 resident=29440 free_bytes=11960385536 total_bytes=12449546240 per_slot=107788 shared_bytes=98416 single_wave=1
bwgpu: 524 coded slices (tables 0, 4, 6, 7,), 369 + 352 code bits a ticket
NVIDIA_RESOURCE_PROBE_OK resident=29440
NVIDIA_RESOURCE_SELECTED block=640 engine=e2_bw_nvidia_avx2_0.4
bw_runner: progress 349/32841 (1%), restored runtime 0 s
bw_runner: progress 2287/32841 (6%), restored runtime 2243 s
bw_runner: progress 4381/32841 (13%), restored runtime 2304 s
bw_runner: progress 5021/32841 (15%), restored runtime 2365 s
bw_runner: progress 5506/32841 (16%), restored runtime 2426 s
bw_runner: progress 5942/32841 (18%), restored runtime 2486 s
bw_runner: progress 6414/32841 (19%), restored runtime 2547 s
bw_runner: progress 6923/32841 (21%), restored runtime 2607 s
bw_runner: progress 7551/32841 (22%), restored runtime 2668 s
bw_runner: progress 8203/32841 (24%), restored runtime 2729 s
bw_runner: progress 9142/32841 (27%), restored runtime 2789 s
bw_runner: progress 10305/32841 (31%), restored runtime 2850 s
bw_runner: progress 11930/32841 (36%), restored runtime 2910 s
bw_runner: progress 14091/32841 (42%), restored runtime 2971 s
bw_runner: progress 16769/32841 (51%), restored runtime 3032 s
bw_runner: progress 19787/32841 (60%), restored runtime 3093 s
bw_runner: progress 22710/32841 (69%), restored runtime 3153 s
bw_runner: progress 25060/32841 (76%), restored runtime 3214 s
bw_runner: progress 26645/32841 (81%), restored runtime 3275 s
bw_runner: progress 27635/32841 (84%), restored runtime 3335 s
bw_runner: progress 28159/32841 (85%), restored runtime 3396 s
bw_runner: progress 28458/32841 (86%), restored runtime 3456 s
bw_runner: progress 28640/32841 (87%), restored runtime 3517 s
bw_runner: progress 28710/32841 (87%), restored runtime 3578 s
bw_runner: progress 28771/32841 (87%), restored runtime 3638 s
bw_runner: progress 28913/32841 (88%), restored runtime 3699 s
bw_runner: progress 29140/32841 (88%), restored runtime 3760 s
bw_runner: progress 29380/32841 (89%), restored runtime 3820 s
bw_runner: progress 29563/32841 (90%), restored runtime 3881 s
bw_runner: progress 29709/32841 (90%), restored runtime 3942 s
bw_runner: progress 29908/32841 (91%), restored runtime 4002 s
bw_runner: progress 30127/32841 (91%), restored runtime 4063 s
bw_runner: progress 30422/32841 (92%), restored runtime 4124 s
bw_runner: progress 30756/32841 (93%), restored runtime 4184 s
bw_runner: progress 31127/32841 (94%), restored runtime 4245 s
bw_runner: progress 31497/32841 (95%), restored runtime 4306 s
bw_runner: progress 31776/32841 (96%), restored runtime 4366 s
bw_runner: progress 31962/32841 (97%), restored runtime 4427 s
bw_runner: progress 32084/32841 (97%), restored runtime 4488 s
bw_runner: progress 32168/32841 (97%), restored runtime 4548 s
bw_runner: progress 32205/32841 (98%), restored runtime 4609 s
bw_runner: progress 32217/32841 (98%), restored runtime 4670 s
bw_runner: progress 32225/32841 (98%), restored runtime 4730 s
bw_runner: progress 32229/32841 (98%), restored runtime 4791 s
bw_runner: progress 32231/32841 (98%), restored runtime 4852 s
bw_runner: progress 32233/32841 (98%), restored runtime 4973 s
bw_runner: progress 32236/32841 (98%), restored runtime 5034 s
bw_runner: progress 32239/32841 (98%), restored runtime 5095 s
bw_runner: progress 32240/32841 (98%), restored runtime 5155 s
bw_runner: progress 32241/32841 (98%), restored runtime 5398 s
bw_runner: progress 32242/32841 (98%), restored runtime 5519 s
bw_runner: progress 32244/32841 (98%), restored runtime 6308 s
2026-10-31 13:07:32 (1067833): wrapper (8.3.26019): starting
2026-10-31 13:07:32 (1067833): wrapper: running ../../projects/boinc.eternityathome.org_eternity/bw_runner_nvidia_common_0.17_linux (--force-interleave --shard-index 385 --shard-count 4096 --force-jitter-start 1869959296 --force-jitters 1870232704 --threads 4 --gpu-slots 0 --node-cap 2000000000 --seconds 0 --exact-endgame 244:50000000 --endgame-probe 4 --endgame-ladder 4:236:1000000000,3:230:6000000000,2:224:64000000000,1:216:80000000000 --ring-breaks 236 --sigma-tail --no-sigma-main --ncon2-direct 1 --mrv-dem-inc 1 --main-fast 1 --min-save 463 --breaks 192,197,201,206,211,216,221,225,229,233,237,241,246,250 --root-allow exploration_allow.txt --seed 6152026 --epoch cloudflare-boinc-explore --tourney tour --ticket-cpu-seconds 0 --runner-tail-fraction 0.005 --runner-idle-handoff 90 --runner-underfill-handoff 90)
2026-10-31 13:07:32 (1067833): wrapper: created child process 1067835
NVIDIA beta: assigned device 0, automatic single resident wave, current-free-memory budget
pipeline: selected scientific engine e2_bw_nvidia_avx2_0.4
bw_runner: resuming, 32244 tickets already completed in this slot will not be recomputed
bw_runner: progress 32244/32841 (98%), restored runtime 6733 s
NVIDIA_RESOURCE_ADMISSION block=640 exit=0
NVIDIA_BETA_PRIORITY inherited_nice=19
[2026-10-31 13:07:32] heuristic colors: {13,16,10}, total edges 122 (ramp scale x1.000)
[2026-10-31 13:07:32] clue mode: FULL-CLUES (5 hints) -> corner-first fill order, heurScale 0.970 (mode default)
[2026-10-31 13:07:32] breaks (14 total, target completion score 466): 192,197,201,206,211,216,221,225,229,233,237,241,246,250
[2026-10-31 13:07:32] P13 SHARD 385/4096 jitterStart=1869959296 : ce worker traite les tickets globaux J0*nroots + 385 + 4096*local (sous-ensemble disjoint)
[2026-10-31 13:07:32] P13 skip-ledger : 32244 tickets terminés chargés (uniques=32244, max=5767797639041) -> jamais recalculés
[2026-10-31 13:07:32] bw starting: 1 threads, seed 6152026, nodeCap 2.0B, heurScale 0.970, minSave 463, resume best 0, seconds 0
[2026-10-31 13:07:32] clue: piece 255 td(2,2) rot 2 -> bu cell (13,2) fill idx 188
[2026-10-31 13:07:32] clue: piece 249 td(2,13) rot 3 -> bu cell (13,13) fill idx 247
[2026-10-31 13:07:32] clue: piece 139 td(8,8) rot 1 -> bu cell (7,8) fill idx 120
[2026-10-31 13:07:32] clue: piece 208 td(13,2) rot 2 -> bu cell (2,2) fill idx 8
[2026-10-31 13:07:32] clue: piece 181 td(13,13) rot 2 -> bu cell (2,13) fill idx 45
[2026-10-31 13:07:32] FORCE_ROOT chargé: 3084 racines, 1870232704 jitters/racine, 1 threads
[2026-10-31 13:07:32] ROOT_ALLOW : 123 cidx autorisés / 3084 racines (32841 lignes lues, 0 hors-bornes ignorées) -> les autres cidx sont sautés (ticket/seed d'origine préservés)
[2026-10-31 13:07:32] TICKET_LOG ARME run_id=861e21df3b34484e cfg_hash=05810c67dbc7af82 dir=startup_probe/640/ticketlog (segment immuable v2)
[2026-10-31 13:07:32] HARVEST463 dir=startup_probe/640/harvest (board vérifié score>=463, identité SHA256)
[2026-10-31 13:07:32] table sizes: corners 16 leftSides 56 bottomSides 56 topSides 1288 rightNoBrk 56 rightBrk 1288 midNoBrk 764 midBrk 34380
[2026-10-31 13:07:32] ring-break tables (from fill depth 236, 8 border cells): corners 720 leftSides 2520 bottomSides 2520 topSides 2520 rightBrk 2520
[2026-10-31 13:07:32] special[0] (cell of clue 255): 1 entries
[2026-10-31 13:07:32] special[1] (west-nb of clue 255): 45 entries
[2026-10-31 13:07:32] special[2] (south-nb of clue 255): 44 entries
[2026-10-31 13:07:32] special[3] (cell of clue 249): 1 entries
[2026-10-31 13:07:32] special[4] (west-nb of clue 249): 2025 entries
[2026-10-31 13:07:32] special[5] (south-nb of clue 249): 2115 entries
[2026-10-31 13:07:32] special[6] (cell of clue 139): 1 entries
[2026-10-31 13:07:32] special[7] (west-nb of clue 139): 44 entries
[2026-10-31 13:07:32] special[8] (south-nb of clue 139): 45 entries
[2026-10-31 13:07:32] special[9] (cell of clue 208): 1 entries
[2026-10-31 13:07:32] special[10] (west-nb of clue 208): 45 entries
[2026-10-31 13:07:32] special[11] (south-nb of clue 208): 46 entries
[2026-10-31 13:07:32] special[12] (cell of clue 181): 1 entries
[2026-10-31 13:07:32] special[13] (west-nb of clue 181): 44 entries
[2026-10-31 13:07:32] special[14] (south-nb of clue 181): 45 entries
[2026-10-31 13:07:32] GPU: tables in left-major slice order: 3,7,11,17 (the others bottom-major)
[2026-10-31 13:07:32] GPU: table 6 (23 deep cells): 234 of 267 slices' no-break groups described
[2026-10-31 13:07:32] GPU: table 7 (41 deep cells): 234 of 493 slices' no-break groups described
bwgpu: frontier of 22 entries (6 groups)
bwgpu: offsets overlay of 4 words (room 228): depths 45 120 188; 98416 bytes of shared memory a block
bwgpu: break test off (no candidate of a scan's range can have too many breaks)
bwgpu: deep region from depth 192; not described (the other region's tables): 196
NVIDIA_BETA slots=640 resident=29440 free_bytes=8103198720 total_bytes=12449546240 per_slot=107788 shared_bytes=98416 single_wave=1
bwgpu: 524 coded slices (tables 0, 4, 6, 7,), 369 + 352 code bits a ticket
NVIDIA_RESOURCE_PROBE_OK resident=29440
NVIDIA_RESOURCE_SELECTED block=640 engine=e2_bw_nvidia_avx2_0.4
bw_runner: progress 32245/32841 (98%), restored runtime 6733 s
bw_runner: progress 32246/32841 (98%), restored runtime 10132 s
bw_runner: progress 32247/32841 (98%), restored runtime 10193 s
bw_runner: progress 32248/32841 (98%), restored runtime 10375 s
bw_runner: progress 32250/32841 (98%), restored runtime 10436 s
bw_runner: progress 32251/32841 (98%), restored runtime 10496 s
bw_runner: progress 32252/32841 (98%), restored runtime 10860 s
bw_runner: progress 32253/32841 (98%), restored runtime 11224 s
bw_runner: progress 32254/32841 (98%), restored runtime 11285 s
|
|
Send message Joined: 23 Sep 26 Posts: 228 Credit: 44,185,220 RAC: 1,478,559 |
In reply to Magiceye04's message of 31 Oct 2026: Thanks Benjamin, Thank you Magiceye04, your log is exactly what I needed! It shows the task isn't frozen, but it's very close to it: the tickets finish one by one, sometimes with very long gaps. Your task still has 587 tickets to do. The app can only hand back the last tickets once 512 or fewer remain, and these tickets have no time limit. That's very likely why it crawls. I'm reviewing this now. For your task, both choices are fine. Since your second task keeps the GPU busy, this one only uses CPU cores, so you can leave it running and hope to get its credit, but it may take many more hours and I can't promise that it will finish. If you abort it, it won't get credit, but that's completely understandable. |
|
New member Send message Joined: 29 Oct 26 Posts: 3 Credit: 2,265,119 RAC: 67,661
|
It finished after some more hours, but as invalid. No problem for me, as it is the only one so far. https://boinc.eternityathome.org/eternity/result.php?resultid=4813585 |
|
Send message Joined: 23 Sep 26 Posts: 228 Credit: 44,185,220 RAC: 1,478,559 |
In reply to Magiceye04's message of 31 Oct 2026: It finished after some more hours, but as invalid. No problem for me, as it is the only one so far. Thanks for the link, Magiceye04, and thanks for taking it so well. I'm sorry: that's almost 13 hours of work. I checked, and it wasn't caused by your computer or by running two tasks. It's on our side. Another volunteer had already returned this task, but our validator had a brief hiccup, so BOINC sent you a second copy to be safe. We hadn't planned for this case: when two graphics cards work on the same task, they never stop at exactly the same place, and our check refuses the second copy when they differ. That check is too strict. It affected a bit over 1% of GPU results over the last 3 days. I can't make changes during the sprint, but it's on my list for after it. Your task also helped: it stopped as soon as 1% of its tickets were left, which confirms what I explained about those long endings. |
|
Send message Joined: 23 Sep 26 Posts: 228 Credit: 44,185,220 RAC: 1,478,559 |
@Magiceye04: a follow-up on your second copy that was rejected: the cause was a slowdown of our validator, which made BOINC send a safety copy that could never match. We moved the credit database to a much lighter system on 3 Nov, and these cases fell from about 190 per day to 3 per day. The long endings are fixed too (version 2.81, see News), so a copy now also ends much sooner. Sorry again for those 13 hours. |
