nil pointer on core install

[trevjonez]

command: arduino-cli core install rambo:avr --debug

INFO[0000] Initiating configuration                     
INFO[0000] Unserializing configurations from /Users/tjones/go/bin/.cli-config.yml 
WARN[0000] Error reading config, using default configuration  error="open /Users/tjones/go/bin/.cli-config.yml: no such file or directory"
WARN[0000] Did not manage to get config file, using default configuration  error="open /Users/tjones/go/bin/.cli-config.yml: no such file or directory"
INFO[0000] Checking if CLI is Bundled into the IDE      
INFO[0000] Candidate IDE Directory: /Users/tjones/go/bin 
INFO[0000] CLI is not bundled into the IDE              
INFO[0000] Configuration set                            
INFO[0000] Starting root command preparation (`arduino`) 
INFO[0000] Formatter set                                
INFO[0000] Executing `arduino core download`            
INFO[0000] Package manager will scan only managed hardware folder 
INFO[0000] Initializing package manager                 
INFO[0000] Loading hardware from: /Users/tjones/Library/arduino15/packages 
INFO[0000] Loading package arduino from: /Users/tjones/Library/arduino15/packages/arduino/hardware 
INFO[0000] Loaded platform                               platform="arduino:avr@1.6.23"
INFO[0000] Checking existence of 'tools' path: /Users/tjones/Library/arduino15/packages/arduino/tools 
INFO[0000] Loading tools from dir: /Users/tjones/Library/arduino15/packages/arduino/tools 
INFO[0000] Loaded tool                                   tool="arduino:arduinoOTA@1.2.1"
INFO[0000] Loaded tool                                   tool="arduino:avr-gcc@5.4.0-atmel3.6.1-arduino2"
INFO[0000] Loaded tool                                   tool="arduino:avrdude@6.3.0-arduino14"
INFO[0000] Loading package builtin from: /Users/tjones/Library/arduino15/packages/builtin 
INFO[0000] Checking existence of 'tools' path: /Users/tjones/Library/arduino15/packages/builtin/tools 
INFO[0000] Loading tools from dir: /Users/tjones/Library/arduino15/packages/builtin/tools 
INFO[0000] Loaded tool                                   tool="builtin:ctags@5.8-arduino11"
INFO[0000] Loading package rambo from: /Users/tjones/Library/arduino15/packages/rambo/hardware 
INFO[0000] Loaded platform                               platform="rambo:avr@1.0.1"
Downloading rambo:avr@1.0.1...
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x40 pc=0x42fbdee]

goroutine 1 [running]:
github.com/arduino/arduino-cli/arduino/resources.(*DownloadResource).ArchivePath(0x0, 0xc0001ed5f0, 0x4d15a10, 0xc000055800, 0x0)
        /Users/tjones/go/src/github.com/arduino/arduino-cli/arduino/resources/helpers.go:31 +0x2e
github.com/arduino/arduino-cli/arduino/resources.(*DownloadResource).IsCached(0x0, 0xc0001ed5f0, 0x40c5e60, 0x4d15a10, 0x0)
        /Users/tjones/go/src/github.com/arduino/arduino-cli/arduino/resources/helpers.go:40 +0x39
github.com/arduino/arduino-cli/arduino/resources.(*DownloadResource).TestLocalArchiveIntegrity(0x0, 0xc0001ed5f0, 0xc0001ac000, 0x0, 0xc0001d1b60)
        /Users/tjones/go/src/github.com/arduino/arduino-cli/arduino/resources/checksums.go:92 +0x3c
github.com/arduino/arduino-cli/arduino/resources.(*DownloadResource).Download(0x0, 0xc0001ed5f0, 0x1e, 0x467c420, 0xc0003b6140)
        /Users/tjones/go/src/github.com/arduino/arduino-cli/arduino/resources/helpers.go:49 +0x4d
github.com/arduino/arduino-cli/arduino/cores/packagemanager.(*PackageManager).DownloadPlatformRelease(0xc000202480, 0xc0002d98f0, 0x0, 0x0, 0xf)
        /Users/tjones/go/src/github.com/arduino/arduino-cli/arduino/cores/packagemanager/download.go:116 +0x3c
github.com/arduino/arduino-cli/commands/core.downloadPlatformByRef(0xc000202480, 0xc0001f0c90)
        /Users/tjones/go/src/github.com/arduino/arduino-cli/commands/core/download.go:79 +0x243
github.com/arduino/arduino-cli/commands/core.runInstallCommand(0xc00020c780, 0xc0001e6aa0, 0x1, 0x2)
        /Users/tjones/go/src/github.com/arduino/arduino-cli/commands/core/install.go:53 +0xd3
github.com/arduino/arduino-cli/vendor/github.com/spf13/cobra.(*Command).execute(0xc00020c780, 0xc0001e6a40, 0x2, 0x2, 0xc00020c780, 0xc0001e6a40)
        /Users/tjones/go/src/github.com/arduino/arduino-cli/vendor/github.com/spf13/cobra/command.go:760 +0x2cc
github.com/arduino/arduino-cli/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0xc0001f8780, 0xc00020d680, 0xc00020c280, 0xc0001f9900)
        /Users/tjones/go/src/github.com/arduino/arduino-cli/vendor/github.com/spf13/cobra/command.go:846 +0x2fd
github.com/arduino/arduino-cli/vendor/github.com/spf13/cobra.(*Command).Execute(0xc0001f8780, 0x0, 0xc0001d1f88)
        /Users/tjones/go/src/github.com/arduino/arduino-cli/vendor/github.com/spf13/cobra/command.go:794 +0x2b
main.main()
        /Users/tjones/go/src/github.com/arduino/arduino-cli/main.go:30 +0x27

.cli-config.yml from current working dir:

board_manager:
  additional_urls:
    - https://raw.githubusercontent.com/ultimachine/ArduinoAddons/master/package_ultimachine_index.json 

Looks like the current dir is not considered for the .cli-config.yml and it is expected in the same dir as the arduino-cli executable. Once I cp the cli config over to the expected dir it works fine.

should probably print a more useful error at the least but my naive understanding it seems like it should consider the working dir for loading the cli config since the config is very much something that could be specific to a project and perhaps undesirable as a global config.

[trevjonez]

seens mostly a duplicate of #30 but I think the error should be more useful than an NPE at the least.

[cmaglie]

uhm I can't reproduce this one:

$ arduino-cli core install rambo:avr
Error: package rambo not found
Could not determine platform dependencies

are you using the latest alpha-release? could you check with arduino-cli version?

[trevjonez]

arduino-cli version 0.3.1-alpha.preview

main issue was my not understanding where to put the .cli-config.yml and forgetting to specify where to look via the --config-file command.

Assumption as to a solution to this issue:
Just need to catch the case of not finding the package given the current board manager config?

Edit:

Just to clarify the install is via go get -u github.com/arduino/arduino-cli @ 14ad16c

[cmaglie]

I'd like to reproduce the nil pointer dereference so I tried to run:

arduino-cli core install rambo:avr

without adding the 3rd party URL, but the result is:

Error: package rambo not found
Could not determine platform dependencies

so the cli correctly detects that there is no rambo:avr core available, without nil pointers.
Looking more closely at the stacktrace on your first post it seems that you already had a rambo package in:

INFO[0000] Loading package rambo from: /Users/tjones/Library/arduino15/packages/rambo/hardware 

maybe it was a partial install?

[trevjonez]

I think it was installed via the IDE initially but if I invoke to install without the correct cli config and it is already installed I get this NP

[dilin-MS]

Hi, @cmaglie I met roughly the same issue here. Could you please help to see how to fix it?

I use arduino-cli in Docker for Windows on my Windows 10 Workstation. I use ubuntu:latest image, which acts like linux 64-bits PC. I added additional url in arduino-cli.yaml file and execute arduino-cli core install AZ3166:stm32f4 --debug, but get the same SIGSEGV dump.

arduino-cli version: arduino-cli-0.3.6-alpha.(Download link)
go version: go1.12.1.linux-amd64(Download link)

Steps:

1. /home/builder/.arduino15/arduino-cli.yaml file:

board_manager:
  additional_urls:
    - https://raw.githubusercontent.com/VSChina/azureiotdevkit_tools/master/package_azureboard_index.json

2. Update index

$ arduino-cli core update-index
Updating index: package_index.json downloaded
Updating index: package_azureboard_index.json downloaded

3. Download AZ3166 package

$ arduino-cli core install AZ3166:stm32f4 --debug
INFO[0000] Checking for config file in: /home/builder/.arduino15/arduino-cli.yaml
INFO[0000] Reading configuration from /home/builder/.arduino15/arduino-cli.yaml
INFO[0000] Checking if CLI is Bundled into the IDE
INFO[0000] Candidate IDE Directory: /usr/bin
INFO[0000] CLI is not bundled into the IDE
INFO[0000] Checking for config in: /arduino-cli.yaml
INFO[0000] error loading                                 error="open /arduino-cli.yaml: no such file or directory"
INFO[0000] Checking for config in: /home/arduino-cli.yaml
INFO[0000] error loading                                 error="open /home/arduino-cli.yaml: no such file or directory"
INFO[0000] Checking for config in: /home/builder/arduino-cli.yaml
INFO[0000] error loading                                 error="open /home/builder/arduino-cli.yaml: no such file or directory"
INFO[0000] Checking for config in: /home/builder/.arduino15/arduino-cli.yaml
INFO[0000] Checking for config in: /home/builder/.arduino15/packages/arduino-cli.yaml
INFO[0000] error loading                                 error="open /home/builder/.arduino15/packages/arduino-cli.yaml: no such file or directory"
INFO[0000] Checking for config in: /home/builder/.arduino15/packages/AZ3166/arduino-cli.yaml
INFO[0000] error loading                                 error="open /home/builder/.arduino15/packages/AZ3166/arduino-cli.yaml: no such file or directory"
INFO[0000] Configuration set
INFO[0000] arduino-cli-0.3.6-alpha.preview
INFO[0000] Starting root command preparation (`arduino`)
INFO[0000] Formatter set
INFO[0000] Executing `arduino core download`
INFO[0000] Package manager will scan only managed hardware folder
INFO[0000] Initializing package manager
INFO[0000] Loading hardware from: /home/builder/.arduino15/packages
INFO[0000] Loading package AZ3166 from: /home/builder/.arduino15/packages/AZ3166/hardware
INFO[0000] Checking existence of 'tools' path: /home/builder/.arduino15/packages/AZ3166/tools
INFO[0000] Loading tools from dir: /home/builder/.arduino15/packages/AZ3166/tools
INFO[0000] Loaded tool                                   tool="AZ3166:arm-none-eabi-gcc@5_4-2016q3"
INFO[0000] Loaded tool                                   tool="AZ3166:openocd@0.10.0"
INFO[0000] Loading package arduino from: /home/builder/.arduino15/packages/arduino
INFO[0000] Checking existence of 'tools' path: /home/builder/.arduino15/packages/arduino/tools
INFO[0000] Loading tools from dir: /home/builder/.arduino15/packages/arduino/tools
INFO[0000] Loaded tool                                   tool="arduino:arduinoSTM32load@2.0.0"
INFO[0000] Loaded tool                                   tool="arduino:arm-none-eabi-gcc@4.8.3-2014q1"
INFO[0000] Loaded tool                                   tool="arduino:dfu-util@0.9.0-arduino1"
INFO[0000] Loading package esp8266 from: /home/builder/.arduino15/packages/esp8266/hardware
INFO[0000] Loaded platform                               platform="esp8266:esp8266@2.5.0"
INFO[0000] Checking existence of 'tools' path: /home/builder/.arduino15/packages/esp8266/tools
INFO[0000] Loading tools from dir: /home/builder/.arduino15/packages/esp8266/tools
INFO[0000] Loaded tool                                   tool="esp8266:esptool@2.5.0-3-20ed2b9"
INFO[0000] Loaded tool                                   tool="esp8266:mkspiffs@2.5.0-3-20ed2b9"
INFO[0000] Loaded tool                                   tool="esp8266:xtensa-lx106-elf-gcc@2.5.0-3-20ed2b9"
WARN[0000] Tool already installed                        platform="AZ3166:stm32f4@1.6.1" tool="AZ3166:arm-none-eabi-gcc@5_4-2016q3"
Tool AZ3166:arm-none-eabi-gcc@5_4-2016q3 already installed
WARN[0000] Tool already installed                        platform="AZ3166:stm32f4@1.6.1" tool="AZ3166:openocd@0.10.0"
Tool AZ3166:openocd@0.10.0 already installed
AZ3166:stm32f4@1.6.1 already downloaded
INFO[0000] Installing platform                           platform="AZ3166:stm32f4@1.6.1"
Installing AZ3166:stm32f4@1.6.1...
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0x4fd462]

goroutine 1 [running]:
github.com/arduino/arduino-cli/vendor/github.com/arduino/go-paths-helper.(*Path).Rename(0x0, 0xc4200755c0, 0xc421892ce0, 0x0)
       /home/jenkins/workspace/arduino-cli/src/github.com/arduino/arduino-cli/vendor/github.com/arduino/go-paths-helper/paths.go:211 +0x22
github.com/arduino/arduino-cli/arduino/resources.(*DownloadResource).Install(0xc4202a1770, 0xc420075de0, 0xc420075df0, 0xc4200755c0, 0x0, 0x0)
       /home/jenkins/workspace/arduino-cli/src/github.com/arduino/arduino-cli/arduino/resources/install.go:88 +0x61b
github.com/arduino/arduino-cli/arduino/cores/packagemanager.(*PackageManager).InstallPlatform(0xc4200a4b80, 0xc4202d0690, 0x0, 0x0)
       /home/jenkins/workspace/arduino-cli/src/github.com/arduino/arduino-cli/arduino/cores/packagemanager/install_uninstall.go:33 +0x108
github.com/arduino/arduino-cli/commands/core.installPlatform(0xc4200a4b80, 0xc4202d0690, 0xc420075330, 0x2, 0x2)
       /home/jenkins/workspace/arduino-cli/src/github.com/arduino/arduino-cli/commands/core/install.go:110 +0x5d2
github.com/arduino/arduino-cli/commands/core.installPlatformByRef(0xc4200a4b80, 0xc42016ced0)
       /home/jenkins/workspace/arduino-cli/src/github.com/arduino/arduino-cli/commands/core/install.go:66 +0x7e
github.com/arduino/arduino-cli/commands/core.runInstallCommand(0xc42019f400, 0xc420105680, 0x1, 0x2)
       /home/jenkins/workspace/arduino-cli/src/github.com/arduino/arduino-cli/commands/core/install.go:53 +0xcb
github.com/arduino/arduino-cli/vendor/github.com/spf13/cobra.(*Command).execute(0xc42019f400, 0xc420105620, 0x2, 0x2, 0xc42019f400, 0xc420105620)
       /home/jenkins/workspace/arduino-cli/src/github.com/arduino/arduino-cli/vendor/github.com/spf13/cobra/command.go:766 +0x2c1
github.com/arduino/arduino-cli/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0xc420153400, 0xc4201a4500, 0xc42019ef00, 0xc42019e780)
       /home/jenkins/workspace/arduino-cli/src/github.com/arduino/arduino-cli/vendor/github.com/spf13/cobra/command.go:852 +0x30a
github.com/arduino/arduino-cli/vendor/github.com/spf13/cobra.(*Command).Execute(0xc420153400, 0x8c7fbc, 0xc420181f78)
       /home/jenkins/workspace/arduino-cli/src/github.com/arduino/arduino-cli/vendor/github.com/spf13/cobra/command.go:800 +0x2b
main.main()
       /home/jenkins/workspace/arduino-cli/src/github.com/arduino/arduino-cli/main.go:30 +0x27

I was able to core install arduino:stm32f4 and esp8266:esp8266@2.5.0 package with no error, but I cannot download AZ3166:stm32f4. I checked and found that many have encountered this issue on different arduino-cli version. Is the bug fixed? I am using the newest arduino-cli(0.3.6-alpha.preview) and the issue occured.

Besides,

1. The package can be core installed smoothly under windows environment.
2. The issue occured when I try it on a linux 64-bits physical machine. Thus the issue has nothing to do with the docker environment.

Any help is appreciated. THX!

[cmaglie]

Hi @dilin-MS, I've analyzed your packages and I think the problem is not the same as the OP.

In your case, the error is caused by your ZIP archive becuase it has the filenames encoded with the backslash \ as a folder separator. The ZIP format, in theory, allows only forward slashes `/' as explained here: https://superuser.com/questions/1382839/zip-files-expand-with-backslashes-on-linux-no-subdirectories

This results on unix/linux in a folder full of files with the \ in the name:

.arduino15/tmp/package-134690621/AZ3166\system\targets\TARGET_STM\TARGET_STM32F4\device\stm32f4xx_hal_smartcard.h
.arduino15/tmp/package-134690621/AZ3166\system\targets\TARGET_STM\TARGET_STM32F4\device\stm32f4xx_hal_spdifrx.h
.arduino15/tmp/package-134690621/AZ3166\system\targets\TARGET_STM\TARGET_STM32F4\device\stm32f4xx_hal_spi.h
.arduino15/tmp/package-134690621/AZ3166\system\targets\TARGET_STM\TARGET_STM32F4\device\stm32f4xx_hal_sram.h
.arduino15/tmp/package-134690621/AZ3166\system\targets\TARGET_STM\TARGET_STM32F4\device\stm32f4xx_hal_tim.h
.arduino15/tmp/package-134690621/AZ3166\system\targets\TARGET_STM\TARGET_STM32F4\device\stm32f4xx_hal_tim_ex.h

just to be clear, inside the folder .arduino15/tmp/package-134690621 there are a bunch of files named:

AZ3166\system\targets\TARGET_STM\TARGET_STM32F4\device\stm32f4xx_hal_smartcard.h
AZ3166\system\targets\TARGET_STM\TARGET_STM32F4\device\stm32f4xx_hal_spdifrx.h
AZ3166\system\targets\TARGET_STM\TARGET_STM32F4\device\stm32f4xx_hal_spi.h
AZ3166\system\targets\TARGET_STM\TARGET_STM32F4\device\stm32f4xx_hal_sram.h
AZ3166\system\targets\TARGET_STM\TARGET_STM32F4\device\stm32f4xx_hal_tim.h
AZ3166\system\targets\TARGET_STM\TARGET_STM32F4\device\stm32f4xx_hal_tim_ex.h

because unix/linux allows backslashes to be part of the name! So a workaround for now would be to re-do the zip archive using forward slashes only.

Now, I'm fixing the crash on the arduino-cli and also print a better error message, PR is coming for this.
I'm also wondering if we should allow ill-formed .zip archives and automatically convert backslashed into forward slashes.

[dilin-MS]

Hi @cmaglie,
if I download the AZ3166 package by wgeting its donwload url directly, and unzip it, the unzip command will have warning but no error. The warning indicates the backslashes problem as you mentioned, saying warning: AZ3166-1.6.1.zip appears to use backslashes as path separators. Noted that the unzip command can still unzip the package properly.

We are using .net api(version earlier than 4.6.1) to zip the package. I figure out that .net api 4.6.1 fix the slash problem.
It may work well if we switch the .net version, but I am suggesting that the arduino-cli should also have the backward compability to support old version windows zip file. After all, there exist many.

Thanks!

[cmaglie]

@dilin-MS there is a build of arduino-cli with a fix here, I've tried and it seems to work now, may you give it a try?

[dilin-MS]

Hi, @cmaglie . I took a try and the build with the fix works like a charm :)
Great job!

[cmaglie]

Reopening since the OP original issue may not be fixed yet.