Beanstalk Longshot test farm
Menu

#1846 perf(vk): the vegetation's indirect mesh draws keep their passes: their arguments converted at the game's barriers (RX_VK_MESH_CONV)

by GummiNaomi closed Open on Forgejo

Commit

Head
2125e5b
Base
main 89c3d21
Approval
approved GummiNaomi, · reviewed

Plan (gpt-6.1-terra)

Compare mesh conversion in Hyrule Field and Market, including vegetation and frame time

In plain words

From kokiri-start, this plan runs the in-game mesh-conversion A/B for two minutes each in Hyrule Field and the Market on Steam Deck, Linux, and Windows with NVIDIA. It compares vegetation screenshots and frame times with main, failing on crashes, device loss, substantial visual differences, or a 15% p99 frame-time regression. Each tester’s game runs take about 15 minutes. The farm cannot set the requested 2560x1440 output or target GPUs by mesh-shader capability, so the no-mesh fallback and its log cannot be guaranteed; the requested frame-trace switch is also unavailable to plans.

Asked for

steamdeck, linux and windows (an NVIDIA card at 2560x1440, and a GPU without mesh shaders if there is one): from kokiri-start, Hyrule Field and Castle Town's market, 2 minutes each, `HOS_AB=RX_VK_MESH_CONV=0,3,60 RX_VK_FRAME_TRACE=1`. Expecting the old way (side 1) slower, as here (+1.7 % in Hyrule Field, +1.3 % in the Market at 2125e5b5's code), the vegetation drawn, and on a GPU without mesh shaders "left as they are (no mesh shaders)" in the log with no change in frame time. This replaces today's run, which ended in `error` before building (the farm's checkout of main failed `git rev-parse --verify HEAD^{commit}` in packs.py's cmd_base) and whose plan also tested `RX_VK_PARALLEL`, which main has removed.

Raw plan 120 lines Download plan.json

  "pass" 
    "crash" false
    "screenshot" 
      "max_diff" 0.1
    
    "frametime_p99_ms" 
      "max_delta_pct" 15
    
    "log_forbidden_hits" 
      "max" 0
    
  
  "probe" 
    
      "kind" "crash"
    
    
      "kind" "frametime"
    
    
      "kind" "log_match"
      "expected" 
      "forbidden" 
        "vk: device lost"
      
    
  
  "matrix" 
  "schema" 3
  "target" 
    
      "name" "steamdeck"
      "tags" 
        "steamdeck"
      
      "max_reports" 2
      "min_reports" 1
    
    
      "name" "linux"
      "tags" 
        "linux"
      
      "max_reports" 2
      "min_reports" 1
    
    
      "name" "windows+nvidia"
      "tags" 
        "windows"
        "nvidia"
      
      "max_reports" 2
      "min_reports" 1
    
  
  "summary" "Compare mesh conversion in Hyrule Field and Market, including vegetation and frame time"
  "baseline" 
    "ref" "main"
  
  "scenario" 
    
      "env" 
        "HOS_AB" "RX_VK_MESH_CONV=0,3,60"
      
      "name" "vegetation-ab"
      "runs" 1
      "step" 
        
          "do" "teleport"
          "to" "HyruleField_Child"
        
        
          "do" "wait"
          "map" "HyruleField_Child"
          "event" "map"
          "timeout_s" 60
        
        
          "do" "shot"
          "name" "field-vegetation"
          "upload" false
          "compare" "baseline"
        
        
          "do" "measure"
          "name" "field"
          "warmup_s" 10
          "duration_s" 120
        
        
          "do" "teleport"
          "to" "Market_Child"
        
        
          "do" "wait"
          "map" "Market_Child"
          "event" "map"
          "timeout_s" 60
        
        
          "do" "shot"
          "name" "market-vegetation"
          "upload" false
          "compare" "baseline"
        
        
          "do" "measure"
          "name" "market"
          "warmup_s" 10
          "duration_s" 120
        
      
      "start" "save:kokiri-start"
      "matrix" true
      "settings" 
    
  

Targets

TargetTierReportsStatus
steamdeckreviewed GummiNaomi none yet 0/1 pass, 0 running, 0 queued
linuxreviewed GummiNaomi 1 0/1 pass, 1 error
windows+nvidiareviewed GummiNaomi 1 2 0/1 pass, 2 error

Reports

ReportCommitTargetTesterResultWhen
rp_be198a2427ecd4952125e5bwindows+nvidia akkusativ/agent-0a71fcerror
rp_d7f479dc6ee4cd042125e5bwindows+nvidia Hidden testererror
rp_811ee7b0654fbc1d2125e5blinux DuckNintendo/linux-4090error
rp_8f87cc5a02ce723827dc849windows CubanLegend/agent-341924error
rp_5dbb3b1eaa40f21627dc849windows Hidden testererror

Jobs

JobCommitTargetStateTesterStartedAttemptsEstimateNote
jb_68f9ed1539bc7004 log2125e5blinux cancelled runHidden tester3 the PR was closed
jb_79acda11aec8b782 log2125e5bsteamdeck cancelled—0 the PR was closed
jb_6ee5c54904d300f3 log2125e5bsteamdeck cancelled—0 the PR was closed
jb_453b2d973d6f2224 log2125e5blinux doneDuckNintendo/linux-40901
jb_4ce02862c7d4ee37 log2125e5bwindows+nvidia done build-baseHidden tester1
jb_d539f07a89d8cf2a log2125e5bwindows+nvidia done build-prakkusativ/agent-0a71fc1
jb_54d255d3de7a7217 log27dc849linux error—5 report: the farm refused the report: the coordinator answered HTTP 400 (BadRequest): {"error":"report.rules[6].status must be one of pass, fail, unsupported, skipped, mismatch"}
jb_349dce85810e7d33 log27dc849windows error—3 report: the farm refused the report: the coordinator answered HTTP 400 (BadRequest): {"error":"report.rules[6].status must be one of pass, fail, unsupported, skipped, mismatch"}
jb_073817d30c068744 log27dc849linux error—4 report: the farm refused the report: the coordinator answered HTTP 400 (BadRequest): {"error":"report.rules[6].status must be one of pass, fail, unsupported, skipped, mismatch"}
jb_dba497165736937a log27dc849steamdeck cancelled—3 superseded by a new push
jb_a46d03c761f47cff log27dc849windows done build-baseHidden tester1
jb_ea682eb59906161c log27dc849linux error—3 report: the farm refused the report: the coordinator answered HTTP 400 (BadRequest): {"error":"report.rules[6].status must be one of pass, fail, unsupported, skipped, mismatch"}
jb_44ab98d0857a9603 log27dc849steamdeck cancelled—3 superseded by a new push
jb_cc5e529df4cc6505 log27dc849windows done build-baseCubanLegend/agent-3419241