RSDK-12022 + RSDK-12276 Provide better progress messaging for hot reloading #5418

michaellee1019 · 2025-10-24T19:49:21Z

The changes in this PR focus on hot reloading and provide more feedback as to what the process is doing and how long it is taking.

I chose the library pterm as it seems like the best golang option for multi-step progress with support to nest some steps inside of others. I did try other dependencies as well and this seems the best and most popular option. I know the CLI generator uses huh but it does not provide the same features such as multiple steps (on different lines) and step nesting. Maybe a separate task could be done to use this new dependency on the generator side.
I made a utility to handle progress tracking across the codebase and called it ProgressManager. Without it the code was gross and had too much state progress management throughout the code. Added unit tests as well.
I removed some conflicting error messages and for now returned the copy progress indicators in this PR. I would like to do another PR that refactors the printing inside of the shell copy code to universally use progress bars provided by pterm.
I also refactored the code so that the build artifact download step actually happens as a post-build step in the code flow.
I refactored addShellService to return whether it actually added the service.

Testing

Success case

michaellee@ROBOT-MYHLX2FTJN working-wheel % viam module reload --part-id 92b25c04-2fc3-444f-9062-15d73f228913
 ✓  Preparing for build... (1s)
 ✓     → Module is registered (0s)                                                                                                                                                                                
 ✓     → Source code archive created (0s)                                                                                                                                                                         
 ✓     → Source code uploaded (1s)                                                                                                                                                                                
 ✓  Building... (1m36s)
 ✓     → Build started (ID: 8015580d-b26d-4ba8-816e-a85cfa9736d0) (0s)                                                                                                                                            
 ✓     → Build completed successfully (1m36s)                                                                                                                                                                     
 ✓  Reloading to part... (27s)
 ✓     → Build artifact downloaded (11s)                                                                                                                                                                          
 ✓     → Shell service already exists (0s)                                                                                                                                                                        
 ✓     → Package uploaded (16s)                                                                                                                                                                                   
 ✓     → Module added to part (0s)

Failed build case

michaellee@ROBOT-MYHLX2FTJN test3 % viam module reload --part-id cfc5404e-e269-425d-b1f9-ad7ce18790e9
Info: Using "app.viam.dev" as base URL value
 ✓  Preparing for build... (1s)
 ✓     → Module is registered (0s)                                                                                                                                                                                
 ✓     → Source code archive created (0s)                                                                                                                                                                         
 ✓     → Source code uploaded (1s)                                                                                                                                                                                
 …  Building...
 ✓     → Build started (ID: 447e7561-9f65-4f8f-bc96-032c26e7645e) (0s)                                                                                                                                            
 ✗     → Build 447e7561-9f65-4f8f-bc96-032c26e7645e failed                                                                                                                                                        
 ✗   Building...
Error: Build "447e7561-9f65-4f8f-bc96-032c26e7645e" failed to complete. Please check the logs below for more information.
Info: viam version
Version ..1-dev.- Git=ff808cb0 API=v0.1.483
Info: echo "::warning::git repository is missing. set 'url' in meta.json to your module's public git repo please"
::warning::git repository is missing. set 'url' in meta.json to your module's public git repo please
Info: curl "https://storage.googleapis.com/viam-packages-e277130c-c24d-428f-a62e-30244ac297bd/e277130c-c24d-428f-a62e-30244ac297bd/packages/test3/5Whuuh5b072iCNG4Pwa2gEYdE49ZyAbm2HfZHxpDmyFbHuYTFe1JrqPSGvSDr8TJ.tar.gz?X-Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Credential=app-service%!s(MISSING)taging-cloud-web-app.iam.gserviceaccount.com%!F(MISSING)20251024%!F(MISSING)auto%!F(MISSING)storage%!F(MISSING)goog4_request&X-Goog-Date=20251024T175643Z&X-Goog-Expires=899&X-Goog-Signature=44bea160e3369434540471cbf5cf25bae004b13e4a518b082547062d9647f45273a1d9501127d35d69edca7834fb9a81a4a1159447d1ce8c6da58345cce2d428f50952ac06bc3811890beb3c9e01f2a3f6e095d9d6e3f953b85f24d6fb0ee4734fd52f61ce7def8bcc94ab7a025258e102ec574ce74b778dcc03d8b33bcd9fa27bf86088708d09b47fff92cd7d08c2ea0a49859a8166d7cc3a638f21a3899a27a0f84eb6a81c65c08a1e472745152c233a8949fabdcebff949609c6eed115ae9be835fb1f4c5b62b81ef4863ee270cb4c7894889cdec94173101e73c549ad6e2e951e50b6817c023e185e094f89ae2fb58859c2b336200ef4f195baa0367994f&X-Goog-SignedHeaders=host" > repo_archive.tar.gz
tar -xzvf repo_archive.tar.gz

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  5077  100  5077    0     0  20389      0 --:--:-- --:--:-- --:--:-- 20389
.github/workflows/deploy.yml
.gitignore
.viam-gen-info
README.md
build.sh
build.sh~
meta.json
requirements.txt
run.sh
setup.sh
src/main.py
src/models/test.py
Info: cd .
viam module build local

Info: Starting build
Info: Starting setup step: "./setup.sh"
Virtualenv found/created. Installing/upgrading Python packages...
Info: Starting build step: "./build.sh"
Error: Error running process "bash": exit status 1

Exited with code exit status 1
Info: Uploading artifacts
Uploading /tmp/module.tar.gz to tmp/module.tar.gz
  No artifact files found at /tmp/module.tar.gz
Total size uploaded: 0 B
Error: Reloading module failed

Prompt to create registry item case

michaellee@ROBOT-MYHLX2FTJN test3 % viam module reload --part-id cfc5404e-e269-425d-b1f9-ad7ce18790e9
Info: Using "app.viam.dev" as base URL value
 …  Preparing for build...
 ⠋     → Ensuring module is registered... (0s)
Error: module not registered in cloud or you lack permissions to edit it.
Info: The reloading process requires the module to first be registered in the cloud. Do you want to proceed with module registration?
Continue: y/n: 
 ✓  Preparing for build... (4s)
 ✓     → Module is registered (0s)                                                                                                                                                                                
 ✓     → Source code archive created (0s)                                                                                                                                                                         
 ✓     → Source code uploaded (1s)                                                                                                                                                                                
 ✓  Building... (1m32s)
 ✓     → Build started (ID: 30f47da2-3542-4487-9cb5-72cc7282ed2c) (0s)                                                                                                                                            
 ✓     → Build completed successfully (1m32s)

Failed to copy to part case

michaellee@ROBOT-MYHLX2FTJN test3 % viam module reload --part-id cfc5404e-e269-425d-b1f9-ad7ce18790e9
Info: Using "app.viam.dev" as base URL value
 ✓  Preparing for build... (2s)
 ✓     → Module is registered (0s)                                                                                                                                                                                
 ✓     → Source code archive created (0s)                                                                                                                                                                         
 ✓     → Source code uploaded (2s)                                                                                                                                                                                
 ✓  Building... (1m18s)
 ✓     → Build started (ID: 570d4966-21a8-4f7d-9bef-4d7e4d16ad04) (0s)                                                                                                                                            
 ✓     → Build completed successfully (1m18s)                                                                                                                                                                     
 …  Reloading to part...
 ✓     → Build artifact downloaded (7s)                                                                                                                                                                           
 ✓     → Shell service already exists (0s)                                                                                                                                                                        
 ⠏     → Uploading package... (30s)2025-10-24T15:31:42.027-0400	DEBUG	shell/copy_local.go:328	unchecked error	{"error": "io: read/write on closed pipe"}
 ✗     → Uploading package...: io: read/write on closed pipe                                                                                                                                                      
 ✗   Reloading to part...
Error: Failed copying to part (~/.viam/packages-local/viam_test3_from_reload-dist-archive.tar.gz): io: read/write on closed pipe

stuqdog

Nice, this looks great. thanks for putting this together! I have a couple comments (mostly minor and optional) though I would like to continue respecting NoProgress or else not advertise it as a thing we do.

stuqdog · 2025-10-28T15:01:10Z

cli/progress_manager.go

+}
+
+// Fail marks a step as failed with an error message.
+func (pm *ProgressManager) Fail(stepID string, err error) error {


(minor, opt) this and FailWithMessage are largely duplicative, might be nice to factor out the shared bits. But also I think this will be touched infrequently enough that it's probably fine to just leave as is.

stuqdog · 2025-10-28T15:13:59Z

cli/progress_manager_test.go

+	}
+
+	elapsed := time.Since(step.startTime)
+	if elapsed < 0 {


(q) would it be useful to instead check that elapsed is greater than 10milis (the sleep time)?

stuqdog · 2025-10-28T15:18:30Z

cli/progress_manager_test.go

+	// We can't easily verify the text was updated since pterm doesn't expose it,
+	// but we can verify no error occurred


It seems like we should be doing some sort of check after this comment?

Oops, this doesn't return an error. I think I will omit the test here since we cant assert anything.

stuqdog · 2025-10-28T15:38:55Z

cli/module_build.go

+		}
 		err = vc.copyFilesToFqdn(
 			part.Part.Fqdn, globalArgs.Debug, false, false, []string{buildPath},
-			dest, logging.NewLogger(reloadVersionPrefix), args.NoProgress)


this was the only location where we used NoProgress for module reloading. I think ideally we should not create the progress manager if NoProgress is true, and have all PM methods return be no-ops if the pm pointer is nil. Barring that, we should probably remove NoProgress from the args type so we don't try to parse it, and make it hidden (though still existent for backwards compatibility) in the viam module reload args list.

Good point. I was too focused on the goal of adding a spinner and forgot about this. Let me reincorporate NoProgress to not make there be a progress spinner.

Ok --no-progress is now respected with the progress manager. I encapsulated it within the progress manager to avoid some gross if/else logic throughout everywhere a status is being set/updated.

stuqdog

nice lgtm!

… into ml/progress-spinner-2

michaellee1019 added 3 commits October 24, 2025 13:07

use pterm

113cdbc

new progress manager and impl with pterm

8ab624a

make lint

aca89d3

viambot added the safe to test This pull request is marked safe to test from a trusted zone label Oct 24, 2025

michaellee1019 changed the title ~~Provide better progress messaging for hot reloading~~ RSDK-12022 + RSDK-12276 Provide better progress messaging for hot reloading Oct 24, 2025

cleanup

5dad26a

viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Oct 24, 2025

Update client.go

364643b

viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Oct 24, 2025

michaellee1019 requested review from njooma and stuqdog October 24, 2025 20:01

fix tests

de5d120

viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Oct 24, 2025

stuqdog requested changes Oct 28, 2025

View reviewed changes

PR comments

1617f70

viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Oct 28, 2025

Merge branch 'main' into ml/progress-spinner-2

35d0682

viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Oct 28, 2025

michaellee1019 requested a review from stuqdog October 28, 2025 19:02

stuqdog approved these changes Oct 29, 2025

View reviewed changes

michaellee1019 added 2 commits October 29, 2025 10:15

fix tests

797f6fb

Merge branch 'ml/progress-spinner-2' of github.com:michaellee1019/rdk…

57851cd

… into ml/progress-spinner-2

viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Oct 29, 2025

make lint

a34ceea

viambot added safe to test This pull request is marked safe to test from a trusted zone and removed safe to test This pull request is marked safe to test from a trusted zone labels Oct 29, 2025

michaellee1019 merged commit b2a3b6c into viamrobotics:main Oct 29, 2025
18 checks passed

michaellee1019 deleted the ml/progress-spinner-2 branch October 29, 2025 15:28

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

RSDK-12022 + RSDK-12276 Provide better progress messaging for hot reloading #5418

RSDK-12022 + RSDK-12276 Provide better progress messaging for hot reloading #5418

Uh oh!

michaellee1019 commented Oct 24, 2025 •

edited

Loading

Uh oh!

stuqdog left a comment

Uh oh!

stuqdog Oct 28, 2025

Uh oh!

stuqdog Oct 28, 2025

Uh oh!

stuqdog Oct 28, 2025

Uh oh!

michaellee1019 Oct 28, 2025

Uh oh!

stuqdog Oct 28, 2025

Uh oh!

michaellee1019 Oct 28, 2025

Uh oh!

michaellee1019 Oct 28, 2025

Uh oh!

stuqdog left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		// We can't easily verify the text was updated since pterm doesn't expose it,
		// but we can verify no error occurred

RSDK-12022 + RSDK-12276 Provide better progress messaging for hot reloading #5418

RSDK-12022 + RSDK-12276 Provide better progress messaging for hot reloading #5418

Uh oh!

Conversation

michaellee1019 commented Oct 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Testing

Uh oh!

stuqdog left a comment

Choose a reason for hiding this comment

Uh oh!

stuqdog Oct 28, 2025

Choose a reason for hiding this comment

Uh oh!

stuqdog Oct 28, 2025

Choose a reason for hiding this comment

Uh oh!

stuqdog Oct 28, 2025

Choose a reason for hiding this comment

Uh oh!

michaellee1019 Oct 28, 2025

Choose a reason for hiding this comment

Uh oh!

stuqdog Oct 28, 2025

Choose a reason for hiding this comment

Uh oh!

michaellee1019 Oct 28, 2025

Choose a reason for hiding this comment

Uh oh!

michaellee1019 Oct 28, 2025

Choose a reason for hiding this comment

Uh oh!

stuqdog left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

michaellee1019 commented Oct 24, 2025 •

edited

Loading