Skip to content

raise error if 'filename' is missing in source dict#5152

Merged
branfosj merged 4 commits intoeasybuilders:developfrom
smoors:source-filename
Apr 1, 2026
Merged

raise error if 'filename' is missing in source dict#5152
branfosj merged 4 commits intoeasybuilders:developfrom
smoors:source-filename

Conversation

@smoors
Copy link
Copy Markdown
Contributor

@smoors smoors commented Mar 30, 2026

avoids ugly crash:

EasyBuild crashed! Please consider reporting a bug, this should not happen...                                                                                                                                                                                                
                                                                                                                                                                                                                                                                             
Traceback (most recent call last):                                                                                                                                                                                                                                           
  File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main                                                                                                                                                                                                     
    return _run_code(code, main_globals, None,                                                                                                                                                                                                                               
  File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code                                                                                                                                                                                                                
    exec(code, run_globals)                                                                                                                                                                                                                                                  
  File "/scratch/brussel/100/vsc10009/easybuild-framework/easybuild/main.py", line 862, in <module>                                                                                                                                                                          
    main_with_hooks()                                                                                                                                                                                                                                                        
  File "/scratch/brussel/100/vsc10009/easybuild-framework/easybuild/main.py", line 847, in main_with_hooks                                                                                                                                                                   
    exit_code: EasyBuildExit = main(args=args, prepared_cfg_data=(init_session_state, eb_go, cfg_settings))                                                                                                                                                                  
  File "/scratch/brussel/100/vsc10009/easybuild-framework/easybuild/main.py", line 798, in main                                                                                                                                                                              
    is_successful = process_eb_args(orig_paths, eb_go, cfg_settings, modtool, testing, init_session_state,                                                                                                                                                                   
  File "/scratch/brussel/100/vsc10009/easybuild-framework/easybuild/main.py", line 614, in process_eb_args                                                                                                                                                                   
    ecs_with_res = build_and_install_software(                                                                                                                                                                                                                               
  File "/scratch/brussel/100/vsc10009/easybuild-framework/easybuild/main.py", line 226, in build_and_install_software                                                                                                                                                        
    raise error                                                                                                                                                                                                                                                              
  File "/scratch/brussel/100/vsc10009/easybuild-framework/easybuild/main.py", line 180, in build_and_install_software                                                                                                                                                        
    (ec_res['success'], app_log, err_msg, err_code) = build_and_install_one(ec, init_env)                                                                                                                                                                                    
  File "/scratch/brussel/100/vsc10009/easybuild-framework/easybuild/framework/easyblock.py", line 5135, in build_and_install_one                                                                                                                                             
    result = app.run_all_steps(run_test_cases=run_test_cases)                                                                                                                                                                                                                
  File "/scratch/brussel/100/vsc10009/easybuild-framework/easybuild/framework/easyblock.py", line 4947, in run_all_steps                                                                                                                                                     
    self.run_step(step_name, step_methods)                                                                                                                                                                                                                                   
  File "/scratch/brussel/100/vsc10009/easybuild-framework/easybuild/framework/easyblock.py", line 4788, in run_step                                                                                                                                                          
    current_method()                                                                                                                                                                                                                                                         
  File "/scratch/brussel/100/vsc10009/easybuild-framework/easybuild/framework/easyblock.py", line 2684, in fetch_step                                                                                                                                                        
    self.fetch_sources(self.cfg['sources'], checksums=self.cfg['checksums'])                                                                                                                                                                                                 
  File "/scratch/brussel/100/vsc10009/easybuild-framework/easybuild/framework/easyblock.py", line 582, in fetch_sources                                                                                                                                                      
    src_spec = self.fetch_source(source, checksum=checksum)                                                                           
  File "/scratch/brussel/100/vsc10009/easybuild-framework/easybuild/framework/easyblock.py", line 538, in fetch_source
    path = self.obtain_file(filename, extension=extension, download_filename=download_filename,                                       
  File "/scratch/brussel/100/vsc10009/easybuild-framework/easybuild/framework/easyblock.py", line 146, in wrapper
    result = func(*args, **kwargs)                                                                                                    
  File "/scratch/brussel/100/vsc10009/easybuild-framework/easybuild/framework/easyblock.py", line 887, in obtain_file                                                                                                                                                            if re.match(r"^(https?|ftp)://", filename):                                                                                                                                                                                                                              
  File "/usr/lib64/python3.9/re.py", line 191, in match                                                                                                                                                                                                                      
    return _compile(pattern, flags).match(string)                                                                                                                                                                                                                            
TypeError: expected string or bytes-like object

@smoors smoors added the bug fix label Mar 30, 2026
@smoors smoors changed the title raise EasyBuildError if is missing in source dict raise EasyBuildError if 'filename' is missing in source dict Mar 30, 2026
@smoors smoors changed the title raise EasyBuildError if 'filename' is missing in source dict raise error if 'filename' is missing in source dict Mar 30, 2026
@branfosj branfosj added this to the next release (5.2.2?) milestone Apr 1, 2026
@branfosj branfosj merged commit 83d9443 into easybuilders:develop Apr 1, 2026
40 checks passed
@Flamefire
Copy link
Copy Markdown
Contributor

Flamefire commented Apr 2, 2026

There can actually be 2 causes: Either filename is missing or it is set to None. If we expect it to be present, why use pop(..., None)?
What about the others? Are they also required? Then I'd remove the "None" and put all in a try-catch KeyError to handle them all.

What was the cause in your case? User forgot to set it? Or some EB method failed to set it?

@smoors
Copy link
Copy Markdown
Contributor Author

smoors commented Apr 2, 2026

There can actually be 2 causes: Either filename is missing or it is set to None. If we expect it to be present, why use pop(..., None)?

there is currently no use case for filename being None, because before this PR EB would crash if it was None. You could also use source.pop('filename'), but then you have to use a try block.

What about the others? Are they also required? Then I'd remove the "None" and put all in a try-catch KeyError to handle them all.

the others are not required.

What was the cause in your case? User forgot to set it? Or some EB method failed to set it?

i forgot to set it in the easyconfig.

@Flamefire
Copy link
Copy Markdown
Contributor

Flamefire commented Apr 2, 2026

there is currently no use case for filename being None,

I was referring to the error message only. One could (wrongly) set it to None and the error will still be

Missing required 'filename' for source

So the question is if we want to differentiate "missing" and "wrong".
However if the only case where we ran into this is missing it in the EC and not some issue in framework then probably not worth it

@smoors
Copy link
Copy Markdown
Contributor Author

smoors commented Apr 2, 2026 •
edited
Loading

So the question is if we want to differentiate "missing" and "wrong".
However if the only case where we ran into this is missing it in the EC and not some issue in framework then probably not worth it

yeah, we could make the error message a bit more clear

EDIT: follow-up in #5156

@smoors smoors deleted the source-filename branch April 2, 2026 09:45
@smoors smoors mentioned this pull request Apr 2, 2026
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@branfosj branfosj branfosj approved these changes

Assignees

No one assigned

Labels

bug fix

Projects

None yet

Milestone

5.3.0

Development

Successfully merging this pull request may close these issues.

3 participants

@smoors @Flamefire @branfosj