QUERY · ISSUE

mip / package.json: Allow 'deps' to come from another repository

openby hmaerkiopened 2023-10-14updated 2023-10-18

Problem

In repo https://github.com/hmaerki/experiment_micropython_mip, I create a mip-package which may be install as follows:

import mip
mip.install("dryer2023", version="main", index="https://hmaerki.github.io/experiment_micropython_mip")

This is the package.json file:

{
    "version": "0.1",
    "hashes": [
        ["dryer2023/statemachine.mpy", "a47f24f2064b"],
        ["dryer2023/__init__.mpy", "c4104085da6d"]
    ],
    "deps": [
        ["umqtt.simple",  "latest"]     <== ERROR A
        ["https://micropython.org/pi/v2/package/6/umqtt.simple/latest.json",  "dummy"]  <== ERROR B
    ]
}

The installation of the hashes works nice.

In the deps section, I would like to install umqtt.simple from another index (the micropython default) but I could not make it working.

ERROR A: Will search in my index (https://hmaerki.github.io/experiment_micropython_mip) which is wrong.
ERROR B: This will correctly download 'latest.json' but then again will try to get the mpy-files from my index which is wrong.

Proposed: Solution

["umqtt.simple", "latest"] should always search in https://micropython.org/pi/v2
["github:org/repo/path/latest.json", "latest"] should search in github:org/repo/path/latest.json

Are there other suggestions/thoughs?

1 comment

ned-pcs · 2023-10-18

Take a look at my work in #676

CANDIDATE · ISSUE

RFC: URL/package.json support in frozen module manifests

openby fill1890opened 2024-09-04updated 2024-09-04

enhancement

Description

manifest.py files currently support installing from micropython-lib with require(); I propose extending the syntax to match that of mip, namely by adding support for a package.json hosted elsewhere (URL, GitHub, or GitLab) and individual .py files by URL.

My initial implementation thoughts are:

Allow github:, gitlab:, and http:// links to be passed to require() in addition to normal package names
Add an optional version=<x> keyword argument to match the mip.install() interface
Port the existing mip functionality either from the module or from mpremote to download modules to a local cache directory during manifest processing
Add these downloaded files to the ManifestFile database so they can be frozen into the firmware with the existing manifest machinery

Please let me know if you have any thoughts/comments/suggestions.

Justification: I'm currently working on a project that uses a number of external libraries I freeze into firmware. I'm using git submodules to pull in library dependencies within the project and package()/modules() to include these into the firmware, however:

I anticipate expanding to multiple firmware variations, which may use different sets of libraries; it may become tedious to set up all required submodules and fetch the correct ones for the firmware being built
This doesn't provide support for freezing a single .py file (unlike mip) from a repository without cloning the entire repository. For example, I'd like to include the typing.mpy file in my firmware for the libraries I use, but I would rather not include the entire repository for it as a submodule.

Code Size

No response

Implementation

I intend to implement this feature and would submit a Pull Request if desirable

Code of Conduct

Yes, I agree