Beanstalk Longshot test farm
Menu

#1927 perf(vk): an update that changes the translator keeps the driver's pipelines and the notes

by madebyjordan open Open on Forgejo

Commit

Head
73a55f5
Base
main 9dd4132
Run
2
Approval
approved madebyjordan, · reviewed

Plan (gpt-6.1-terra)

Compare startup and gameplay on requested hardware, including cold and warm Linux driver caches

⚠️ Note: The farm cannot seed nightly-622 caches or perform the requested update, so it cannot verify the carry log, preparation time, or carried-cache costs against a fresh cache.

⚠️ Note: Cold and warm driver-cache scenarios run only on Linux; Windows gets the ordinary startup and play comparison. RAM is measured during gameplay, not peak preparation memory.

In plain words

The requested update from a nightly-622-prepared folder cannot be reproduced, so this alternative compares ordinary startup and play against main on Linux NVIDIA, Linux RADV, Windows NVIDIA, and Steam Deck. Linux runs also compare starts with cold and warm driver caches; these are not fresh game caches. The plan measures startup time and gameplay RAM, and fails on crashes, substantial startup or frame-time regressions, or a changed gameplay picture. It takes about 30–35 minutes per Linux tester and about 10–12 minutes per Windows tester, excluding builds.

Asked for

Please test this on Linux (NVIDIA and AMD RADV), Windows NVIDIA and the Steam Deck: a game folder prepared by nightly-622, then updated to this branch, which is a real translator change. Expect the log to say `another translator's ... carried`, a preparation of minutes rather than a full compile, the right picture in play, and no crash. Please also report RAM and the time to the first frame against a fresh cache where the run can. --- and another request --- test it anyway

Raw plan 174 lines Download plan.json

  "pass" 
    "crash" false
    "screenshot" 
      "max_diff" 0.05
    
    "boot_time_s" 
      "max_delta_pct" 25
    
    "frametime_p99_ms" 
      "max_delta_pct" 20
    
  
  "probe" 
    
      "kind" "crash"
    
    
      "kind" "boot_time"
    
    
      "kind" "frametime"
    
    
      "kind" "process"
    
    
      "kind" "shader_compile"
    
  
  "matrix" 
  "schema" 9
  "target" 
    
      "name" "linux+nvidia"
      "tags" 
        "linux"
        "nvidia"
      
      "max_reports" 2
      "min_reports" 1
    
    
      "name" "linux+radv"
      "tags" 
        "linux"
        "radv"
      
      "max_reports" 2
      "min_reports" 1
    
    
      "name" "windows+nvidia"
      "tags" 
        "windows"
        "nvidia"
      
      "max_reports" 2
      "min_reports" 1
    
    
      "name" "steamdeck"
      "tags" 
        "steamdeck"
      
      "max_reports" 2
      "min_reports" 1
    
  
  "summary" "Compare startup and gameplay on requested hardware, including cold and warm Linux driver caches"
  "baseline" 
    "ref" "main"
  
  "scenario" 
    
      "env" 
      "name" "ordinary-play"
      "runs" 1
      "step" 
        
          "do" "teleport"
          "to" "HyruleField_Child"
        
        
          "do" "wait"
          "map" "HyruleField_Child"
          "event" "map"
          "timeout_s" 120
        
        
          "do" "shot"
          "name" "field"
          "upload" false
          "compare" "baseline"
        
        
          "do" "measure"
          "name" "field"
          "warmup_s" 5
          "duration_s" 20
        
      
      "start" "save:kokiri-start"
      "matrix" true
      "settings" 
    
    
      "env" 
      "name" "driver-cold"
      "runs" 1
      "step" 
        
          "do" "teleport"
          "to" "KokiriForest_Child"
        
        
          "do" "wait"
          "map" "KokiriForest_Child"
          "event" "map"
          "timeout_s" 120
        
        
          "do" "shot"
          "name" "forest-cold"
          "upload" false
          "compare" "baseline"
        
        
          "do" "measure"
          "name" "forest"
          "warmup_s" 5
          "duration_s" 20
        
      
      "cache" "driver_cold"
      "start" "save:kokiri-start"
      "matrix" true
      "settings" 
    
    
      "env" 
      "name" "driver-warm"
      "runs" 1
      "step" 
        
          "do" "teleport"
          "to" "KokiriForest_Child"
        
        
          "do" "wait"
          "map" "KokiriForest_Child"
          "event" "map"
          "timeout_s" 120
        
        
          "do" "shot"
          "name" "forest-warm"
          "upload" false
          "compare" "baseline"
        
        
          "do" "measure"
          "name" "forest"
          "warmup_s" 5
          "duration_s" 20
        
      
      "cache" "driver_warm"
      "start" "save:kokiri-start"
      "matrix" true
      "settings" 
    
  

Targets

TargetTierReportsStatus
linux+nvidiareviewed madebyjordan none yet 0/1 pass, 2 running, 0 queued
linux+radvreviewed madebyjordan none yet 0/1 pass, 2 running, 0 queued
windows+nvidiareviewed madebyjordan none yet 0/1 pass, 2 running, 0 queued
steamdeckreviewed madebyjordan none yet 0/1 pass, 1 running, 1 queued

Reports

No reports yet. Each tester posts one when its job ends.

Jobs

Estimate: done in about 1 h 30 min, from the queue and the testers online now; some jobs can't start yet.

JobCommitTargetStateTesterStartedAttemptsEstimateNote
jb_b691a3c2ef6acc22 log73a55f5linux+nvidia runningbuild-pr42%42%Hidden tester1 done in about 1 h 30 min
jb_08e815279ec9d9fb log73a55f5linux+radv runningbuild-base10%10%Simon-Olivier/agent-1b2b4f1 done in about 55 min
jb_3d72a662289e836f log73a55f5linux+nvidia runningbuild-pr42%42%Floppy/agent-06957c1 done in about 35 min
jb_1d864812d3a25e97 log73a55f5steamdeck runningbuild-base10%10%Wrath/agent-0e17b11 done in about 55 min
jb_555b2f5469a25716 log73a55f5linux+radv runningbuild-base10%10%manasuazo/agent-03253d1 done in about 55 min
jb_69b4cac0a639ddfa log73a55f5windows+nvidia runningrun66%66%enum/agent-5773ed1 done in about 15 min
jb_00b9718bf640ac76 log73a55f5windows+nvidia runningbuild-pr42%42%Samsheperd/agent-a9f5121 done in about 60 min
jb_92a59befec09548d73a55f5steamdeck queued—0 No tester online can take it now.1 tester matches `steamdeck` online, but 1 ran or skipped this job already.