| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
fix(deps): override semgrep's pyjwt pin and databricks' oauthlib cap (#1859) Clears the remaining Dependabot alerts, which were blocked by upstream pins with no patched parent release available. - pyjwt 2.13.0 -> 2.15.1 in security-scan-tools.txt: semgrep (latest 1.178.0) pins pyjwt~=2.13.0. Compile with a new security-scan-tools-overrides.txt (pyjwt[crypto]>=2.15.1) and install with --no-deps in security-scan.yml, since pip's resolver rejects the out-of-range version. Every transitive dep is already hash-pinned. - oauthlib 3.3.1 -> 4.0.0 in uv.lock: databricks-sql-connector (latest 4.6.0) caps oauthlib<4.0.0. Add [tool.uv] override-dependencies. The connector only uses WebApplicationClient methods, verified unchanged on 4.0.0. | 18 天前 | |
Fix CurrencyNormalizer dropping K/M/B magnitude suffixes (#1854) * fix(normalize): preserve K/M/B magnitude suffixes in currency parsing * docs(normalize): add K/M/B magnitude suffix example * fix(normalize): reject malformed amounts and support bare leading decimals Addresses PR review feedback: the fallback parser inflated bare leading-decimal amounts like ".5" (parsed as 5 instead of 0.5) and silently accepted malformed numbers like "$1.2.3" (parsed as 1.2 instead of None). * fix(deps): bump urllib3 to 2.8.0 to resolve pip-audit CVEs * fix(deps): bump virtualenv to 21.9.0 to resolve pip-audit CVEs | 19 天前 | |
fix(ci): drop unpinnable benchmarks/requirements.txt install (#1345) * fix(ci): drop unpinnable benchmarks/requirements.txt install Scorecard flagged this pip install as unpinned-by-hash (#6082). Can't hash-pin it - benchmarks/requirements.txt doesn't exist in this repo, so there's nothing to compile a lockfile from. Dropping it instead of leaving it unpinned: the job already fails on the next real step (benchmarks/benchmarks_runner.py, also missing), so this line wasn't doing anything useful to begin with. * fix(ci): hash-pin the spacy model download in benchmark.yml Qodo review on this PR: dropping the benchmarks/requirements.txt install (the previous failure point) let the job actually reach `python -m spacy download en_core_web_sm`, which fetches an unpinned, unhashed wheel from spacy-models' GitHub releases - undoing the point of this PR by exposing a real unpinned-install path instead of a dead one. Replaced with a hash-pinned direct-URL entry in benchmark-extra.in/.txt for en_core_web_sm-3.8.0 (matches the spacy==3.8.15 already pinned in base-deps.txt). uv independently computed the same sha256 I got via a manual curl+sha256 of the release asset, and a --require-hashes dry-run install verifies clean. | 1 个月前 | |
This is resubmit of the pr for issue feat(integrations): add Google ADK support (#1312) Adds Google ADK (Agent Development Kit) support to Semantica. `semantica_kg_tools()` and `semantica_decision_tools()` expose entity/relation extraction, graph updates, and decision recording as ADK `FunctionTool`s. `SemanticaSessionService` implements ADK session storage on top of a Semantica `ContextGraph`, so session state, events, and knowledge graph data can live in the same graph instead of keeping sessions in memory. There were also a number of dependency and CI fixes needed to get the integration working reliably. `google-adk` is pinned to a range that avoids the CI `websockets` conflict, the deprecated `pinecone-client` dependency was replaced with `pinecone`, Windows-only dependencies now have the appropriate platform markers, and `requirements-ci.txt` was regenerated to match. A `pip-audit` pass also required updates to `google-adk` and `starlette` for known CVEs. Some unrelated `pyproject.toml` changes had slipped in during rebases, so the previous version, dependency bounds, `ingest-sap`/LangChain entries, and package-data settings were restored. A few bugs in the initial ADK implementation were fixed during review: * `extract_relations()` was calling `RelationExtractor.extract_entities()`, which doesn't exist on that extractor. The failure was being caught and returned in the tool's `error` field, leaving callers with an empty relation list. It now calls the correct extraction path. * The repo's top-level `mcp/` package shadowed the third-party `mcp` package imported by `google.adk`, causing `google.adk` imports to fail from a normal repo checkout. The local package was moved to `semantica_mcp/mcp/`. The MCP move needed a follow-up as well. `semantica/cli.py` and four existing tests were still importing from `mcp.*`, and the modules under `semantica_mcp/mcp/` still used the old absolute imports internally. `semantica_mcp` was also missing from the setuptools package include list and had no `__init__.py`, so it wouldn't have been included in an installed package. Those imports and packaging settings are fixed now. The session service and ADK tools also had a few other problems: * `list_sessions()` returned a plain list instead of ADK's `ListSessionsResponse`. The original import for that type doesn't work against the installed `google-adk` package, so it was silently falling back to a stub. `user_id` was also incorrectly required instead of being optional. * Session node IDs were built by joining `app_name`, `user_id`, and `session_id` with unescaped colons, which allowed different identities to produce the same graph node ID. Each component is now encoded before joining. * `kg_tools.py` and `decision_tools.py` each had their own lock registry and default graph instance. Sharing a graph between the two modules therefore didn't share the lock, and using both factories without an explicit graph produced two different defaults. The shared state now lives in one module used by both. * `add_to_graph` had a `TypeError` compatibility fallback that couldn't succeed with the current `RelationExtractor` API and could hide the original extraction error. That fallback was removed. * `append_event` persisted partial streaming events even though ADK's base session service skips them. * `get_session()` ignored its `config` argument, so `num_recent_events` and `after_timestamp` had no effect. * The async session-service methods performed synchronous graph scans while holding a `threading.RLock` on the event loop thread. That work now runs in worker threads with `asyncio.to_thread()` so a slow or contended graph operation doesn't block the loop. --- Co-authored-by: Zohaib Hassnain [109234410+ZohaibHassan16@users.noreply.github.com](mailto:109234410+ZohaibHassan16@users.noreply.github.com) | 1 个月前 | |
fix(ci): hash-pin every pip install for Scorecard Pinned-Dependencies Scorecard's Pinned-Dependencies check requires pip installs to be hash-verified, not just version-pinned - our existing pkg==X.Y.Z pins (and even pip install -r requirements-ci.txt, despite that file already carrying hashes) still scored a 4 because the pin/hash isn't visible on the command line itself. Adds .github/requirements/*.txt: hash-locked files generated via `uv pip compile --generate-hashes` for every pip target that isn't already requirements-ci.txt, covering standalone CI tooling (build, wheel, twine, uv, pip-audit, safety/bandit/semgrep/jq, pip/setuptools bootstrap) and the project's own local-source installs. The latter (`pip install -e ".[explorer]"`, `pip install -e .`) can't be hash-pinned directly since there's nothing to hash for a local source tree; split into `pip install --no-deps -e .` plus a separate hash-pinned install of the actual fetched dependencies instead. Also adds --require-hashes to every `-r requirements-ci.txt` install so hash verification is enforced explicitly rather than only implied by the file's own content. Simplifies the Dockerfile in the process: it now installs from the same pre-generated explorer-extra.txt (copied in at build time) instead of extracting constraints from requirements-ci.txt at build time, which also means setuptools gets its CVE-2025-47273 fix as a side effect of the hash-pinned install rather than a separate upgrade step. benchmark.yml: pip install -r benchmarks/requirements.txt is left unpinned - that directory doesn't exist in this repo, so there's nothing to generate hashes from. Pre-existing breakage, unrelated to this change. | 1 个月前 | |
fix(ci): hash-pin every pip install for Scorecard Pinned-Dependencies Scorecard's Pinned-Dependencies check requires pip installs to be hash-verified, not just version-pinned - our existing pkg==X.Y.Z pins (and even pip install -r requirements-ci.txt, despite that file already carrying hashes) still scored a 4 because the pin/hash isn't visible on the command line itself. Adds .github/requirements/*.txt: hash-locked files generated via `uv pip compile --generate-hashes` for every pip target that isn't already requirements-ci.txt, covering standalone CI tooling (build, wheel, twine, uv, pip-audit, safety/bandit/semgrep/jq, pip/setuptools bootstrap) and the project's own local-source installs. The latter (`pip install -e ".[explorer]"`, `pip install -e .`) can't be hash-pinned directly since there's nothing to hash for a local source tree; split into `pip install --no-deps -e .` plus a separate hash-pinned install of the actual fetched dependencies instead. Also adds --require-hashes to every `-r requirements-ci.txt` install so hash verification is enforced explicitly rather than only implied by the file's own content. Simplifies the Dockerfile in the process: it now installs from the same pre-generated explorer-extra.txt (copied in at build time) instead of extracting constraints from requirements-ci.txt at build time, which also means setuptools gets its CVE-2025-47273 fix as a side effect of the hash-pinned install rather than a separate upgrade step. benchmark.yml: pip install -r benchmarks/requirements.txt is left unpinned - that directory doesn't exist in this repo, so there's nothing to generate hashes from. Pre-existing breakage, unrelated to this change. | 1 个月前 | |
fix(ci): hash-pin every pip install for Scorecard Pinned-Dependencies Scorecard's Pinned-Dependencies check requires pip installs to be hash-verified, not just version-pinned - our existing pkg==X.Y.Z pins (and even pip install -r requirements-ci.txt, despite that file already carrying hashes) still scored a 4 because the pin/hash isn't visible on the command line itself. Adds .github/requirements/*.txt: hash-locked files generated via `uv pip compile --generate-hashes` for every pip target that isn't already requirements-ci.txt, covering standalone CI tooling (build, wheel, twine, uv, pip-audit, safety/bandit/semgrep/jq, pip/setuptools bootstrap) and the project's own local-source installs. The latter (`pip install -e ".[explorer]"`, `pip install -e .`) can't be hash-pinned directly since there's nothing to hash for a local source tree; split into `pip install --no-deps -e .` plus a separate hash-pinned install of the actual fetched dependencies instead. Also adds --require-hashes to every `-r requirements-ci.txt` install so hash verification is enforced explicitly rather than only implied by the file's own content. Simplifies the Dockerfile in the process: it now installs from the same pre-generated explorer-extra.txt (copied in at build time) instead of extracting constraints from requirements-ci.txt at build time, which also means setuptools gets its CVE-2025-47273 fix as a side effect of the hash-pinned install rather than a separate upgrade step. benchmark.yml: pip install -r benchmarks/requirements.txt is left unpinned - that directory doesn't exist in this repo, so there's nothing to generate hashes from. Pre-existing breakage, unrelated to this change. | 1 个月前 | |
fix(ci): hash-pin every pip install for Scorecard Pinned-Dependencies Scorecard's Pinned-Dependencies check requires pip installs to be hash-verified, not just version-pinned - our existing pkg==X.Y.Z pins (and even pip install -r requirements-ci.txt, despite that file already carrying hashes) still scored a 4 because the pin/hash isn't visible on the command line itself. Adds .github/requirements/*.txt: hash-locked files generated via `uv pip compile --generate-hashes` for every pip target that isn't already requirements-ci.txt, covering standalone CI tooling (build, wheel, twine, uv, pip-audit, safety/bandit/semgrep/jq, pip/setuptools bootstrap) and the project's own local-source installs. The latter (`pip install -e ".[explorer]"`, `pip install -e .`) can't be hash-pinned directly since there's nothing to hash for a local source tree; split into `pip install --no-deps -e .` plus a separate hash-pinned install of the actual fetched dependencies instead. Also adds --require-hashes to every `-r requirements-ci.txt` install so hash verification is enforced explicitly rather than only implied by the file's own content. Simplifies the Dockerfile in the process: it now installs from the same pre-generated explorer-extra.txt (copied in at build time) instead of extracting constraints from requirements-ci.txt at build time, which also means setuptools gets its CVE-2025-47273 fix as a side effect of the hash-pinned install rather than a separate upgrade step. benchmark.yml: pip install -r benchmarks/requirements.txt is left unpinned - that directory doesn't exist in this repo, so there's nothing to generate hashes from. Pre-existing breakage, unrelated to this change. | 1 个月前 | |
fix(security): bump checkov to 3.3.16, fix aiohttp CVEs in its lockfile (#1342) Dependabot flagged 12 aiohttp advisories (1 high, rest moderate/low - CVE range covering request smuggling, websocket/parser bugs, cookie/redirect issues) against aiohttp==3.13.5 pinned in checkov.txt. checkov==3.3.1 itself pinned `aiohttp<3.14.0`, which excludes every fixed release; 3.3.16 (latest) relaxes that to `<3.15.0`, so bumping checkov also lets aiohttp resolve to 3.14.3 (fixes all of them). Two alerts remain open, both genuinely blocked upstream rather than something a version bump here can fix: - asteval: checkov 3.3.16 (latest, still) hard-pins asteval==1.0.6 with no range; the fix (1.0.9) is unresolvable without violating checkov's own declared dependency - confirmed via `uv pip compile` refusing to solve it. Needs checkov itself to bump the pin upstream. - ecdsa: 0.19.2 is already the latest release; the Minerva timing-attack advisory has no patched version, since python-ecdsa's maintainers have stated side-channel attacks are out of scope for the project. Both are checkov's own transitive deps, used only for local static IaC analysis in defender-for-devops.yml (no network signing/cloud-auth calls that would actually exercise ecdsa's signing path) - dismissing on GitHub with that reasoning as a separate step. | 1 个月前 | |
fix(deps): bump aiohttp to 3.14.5 to clear the scheduled security scan (#2046) * fix(deps): bump aiohttp to 3.14.5 to clear the scheduled security scan aiohttp==3.14.3 carries CVE-2026-111063, CVE-2026-111064 and GHSA-2g87-pp6c-29x5, which fails the required security-scan check and blocks every merge. First fixed release is 3.14.4; bumped to 3.14.5, the latest release with no known advisories, per the note above IGNORED_VULN_IDS in .github/workflows/security-scan.yml. Refs #2045 * fix(deps): bump yarl to 1.25.1 for aiohttp 3.14.5 aiohttp 3.14.5 requires yarl>=1.25.1 and requirements-ci.txt pinned yarl==1.24.5, so the pinned install failed with ResolutionImpossible. uv.lock was already on yarl 1.25.1; this brings the requirements files in line. yarl 1.25.1 has no open advisories. --------- Co-authored-by: pkupt <yoaldrinjoseph@gmail.com> Co-authored-by: Kaif <98801504+KaifAhmad1@users.noreply.github.com> | 4 小时前 | |
Fix CurrencyNormalizer dropping K/M/B magnitude suffixes (#1854) * fix(normalize): preserve K/M/B magnitude suffixes in currency parsing * docs(normalize): add K/M/B magnitude suffix example * fix(normalize): reject malformed amounts and support bare leading decimals Addresses PR review feedback: the fallback parser inflated bare leading-decimal amounts like ".5" (parsed as 5 instead of 0.5) and silently accepted malformed numbers like "$1.2.3" (parsed as 1.2 instead of None). * fix(deps): bump urllib3 to 2.8.0 to resolve pip-audit CVEs * fix(deps): bump virtualenv to 21.9.0 to resolve pip-audit CVEs | 19 天前 | |
Fix CurrencyNormalizer dropping K/M/B magnitude suffixes (#1854) * fix(normalize): preserve K/M/B magnitude suffixes in currency parsing * docs(normalize): add K/M/B magnitude suffix example * fix(normalize): reject malformed amounts and support bare leading decimals Addresses PR review feedback: the fallback parser inflated bare leading-decimal amounts like ".5" (parsed as 5 instead of 0.5) and silently accepted malformed numbers like "$1.2.3" (parsed as 1.2 instead of None). * fix(deps): bump urllib3 to 2.8.0 to resolve pip-audit CVEs * fix(deps): bump virtualenv to 21.9.0 to resolve pip-audit CVEs | 19 天前 | |
Fix CurrencyNormalizer dropping K/M/B magnitude suffixes (#1854) * fix(normalize): preserve K/M/B magnitude suffixes in currency parsing * docs(normalize): add K/M/B magnitude suffix example * fix(normalize): reject malformed amounts and support bare leading decimals Addresses PR review feedback: the fallback parser inflated bare leading-decimal amounts like ".5" (parsed as 5 instead of 0.5) and silently accepted malformed numbers like "$1.2.3" (parsed as 1.2 instead of None). * fix(deps): bump urllib3 to 2.8.0 to resolve pip-audit CVEs * fix(deps): bump virtualenv to 21.9.0 to resolve pip-audit CVEs | 19 天前 | |
fix(ci): hash-pin PEP 517 build isolation deps (setuptools, wheel) Qodo review on this PR: `pip install --no-deps -e .` / `pip install --no-deps .` still leaves PEP 517 build isolation on by default, which fetches [build-system] requires (setuptools==84.0.0, wheel==0.48.0) completely outside any hash checking - the --require-hashes installs right next to it didn't cover this at all. Adds .github/requirements/pep517-build.txt, hash-locked to the exact pyproject.toml [build-system] requires, and installs it before every local-source install (Dockerfile, ci.yml, benchmark.yml) with --no-build-isolation so pip reuses those hash-verified copies instead of fetching its own. | 1 个月前 | |
fix(ci): hash-pin PEP 517 build isolation deps (setuptools, wheel) Qodo review on this PR: `pip install --no-deps -e .` / `pip install --no-deps .` still leaves PEP 517 build isolation on by default, which fetches [build-system] requires (setuptools==84.0.0, wheel==0.48.0) completely outside any hash checking - the --require-hashes installs right next to it didn't cover this at all. Adds .github/requirements/pep517-build.txt, hash-locked to the exact pyproject.toml [build-system] requires, and installs it before every local-source install (Dockerfile, ci.yml, benchmark.yml) with --no-build-isolation so pip reuses those hash-verified copies instead of fetching its own. | 1 个月前 | |
fix(ci): hash-pin every pip install for Scorecard Pinned-Dependencies Scorecard's Pinned-Dependencies check requires pip installs to be hash-verified, not just version-pinned - our existing pkg==X.Y.Z pins (and even pip install -r requirements-ci.txt, despite that file already carrying hashes) still scored a 4 because the pin/hash isn't visible on the command line itself. Adds .github/requirements/*.txt: hash-locked files generated via `uv pip compile --generate-hashes` for every pip target that isn't already requirements-ci.txt, covering standalone CI tooling (build, wheel, twine, uv, pip-audit, safety/bandit/semgrep/jq, pip/setuptools bootstrap) and the project's own local-source installs. The latter (`pip install -e ".[explorer]"`, `pip install -e .`) can't be hash-pinned directly since there's nothing to hash for a local source tree; split into `pip install --no-deps -e .` plus a separate hash-pinned install of the actual fetched dependencies instead. Also adds --require-hashes to every `-r requirements-ci.txt` install so hash verification is enforced explicitly rather than only implied by the file's own content. Simplifies the Dockerfile in the process: it now installs from the same pre-generated explorer-extra.txt (copied in at build time) instead of extracting constraints from requirements-ci.txt at build time, which also means setuptools gets its CVE-2025-47273 fix as a side effect of the hash-pinned install rather than a separate upgrade step. benchmark.yml: pip install -r benchmarks/requirements.txt is left unpinned - that directory doesn't exist in this repo, so there's nothing to generate hashes from. Pre-existing breakage, unrelated to this change. | 1 个月前 | |
Fix CurrencyNormalizer dropping K/M/B magnitude suffixes (#1854) * fix(normalize): preserve K/M/B magnitude suffixes in currency parsing * docs(normalize): add K/M/B magnitude suffix example * fix(normalize): reject malformed amounts and support bare leading decimals Addresses PR review feedback: the fallback parser inflated bare leading-decimal amounts like ".5" (parsed as 5 instead of 0.5) and silently accepted malformed numbers like "$1.2.3" (parsed as 1.2 instead of None). * fix(deps): bump urllib3 to 2.8.0 to resolve pip-audit CVEs * fix(deps): bump virtualenv to 21.9.0 to resolve pip-audit CVEs | 19 天前 | |
fix(ci): hash-pin every pip install for Scorecard Pinned-Dependencies Scorecard's Pinned-Dependencies check requires pip installs to be hash-verified, not just version-pinned - our existing pkg==X.Y.Z pins (and even pip install -r requirements-ci.txt, despite that file already carrying hashes) still scored a 4 because the pin/hash isn't visible on the command line itself. Adds .github/requirements/*.txt: hash-locked files generated via `uv pip compile --generate-hashes` for every pip target that isn't already requirements-ci.txt, covering standalone CI tooling (build, wheel, twine, uv, pip-audit, safety/bandit/semgrep/jq, pip/setuptools bootstrap) and the project's own local-source installs. The latter (`pip install -e ".[explorer]"`, `pip install -e .`) can't be hash-pinned directly since there's nothing to hash for a local source tree; split into `pip install --no-deps -e .` plus a separate hash-pinned install of the actual fetched dependencies instead. Also adds --require-hashes to every `-r requirements-ci.txt` install so hash verification is enforced explicitly rather than only implied by the file's own content. Simplifies the Dockerfile in the process: it now installs from the same pre-generated explorer-extra.txt (copied in at build time) instead of extracting constraints from requirements-ci.txt at build time, which also means setuptools gets its CVE-2025-47273 fix as a side effect of the hash-pinned install rather than a separate upgrade step. benchmark.yml: pip install -r benchmarks/requirements.txt is left unpinned - that directory doesn't exist in this repo, so there's nothing to generate hashes from. Pre-existing breakage, unrelated to this change. | 1 个月前 | |
fix(ci): hash-pin every pip install for Scorecard Pinned-Dependencies Scorecard's Pinned-Dependencies check requires pip installs to be hash-verified, not just version-pinned - our existing pkg==X.Y.Z pins (and even pip install -r requirements-ci.txt, despite that file already carrying hashes) still scored a 4 because the pin/hash isn't visible on the command line itself. Adds .github/requirements/*.txt: hash-locked files generated via `uv pip compile --generate-hashes` for every pip target that isn't already requirements-ci.txt, covering standalone CI tooling (build, wheel, twine, uv, pip-audit, safety/bandit/semgrep/jq, pip/setuptools bootstrap) and the project's own local-source installs. The latter (`pip install -e ".[explorer]"`, `pip install -e .`) can't be hash-pinned directly since there's nothing to hash for a local source tree; split into `pip install --no-deps -e .` plus a separate hash-pinned install of the actual fetched dependencies instead. Also adds --require-hashes to every `-r requirements-ci.txt` install so hash verification is enforced explicitly rather than only implied by the file's own content. Simplifies the Dockerfile in the process: it now installs from the same pre-generated explorer-extra.txt (copied in at build time) instead of extracting constraints from requirements-ci.txt at build time, which also means setuptools gets its CVE-2025-47273 fix as a side effect of the hash-pinned install rather than a separate upgrade step. benchmark.yml: pip install -r benchmarks/requirements.txt is left unpinned - that directory doesn't exist in this repo, so there's nothing to generate hashes from. Pre-existing breakage, unrelated to this change. | 1 个月前 | |
fix(deps): override semgrep's mcp pin to clear GHSA-84m7-p3x7-pcfv (#2049) semgrep (through 1.181.0) pins mcp==1.29.0, which is affected by the unbounded Streamable HTTP session advisory fixed in 1.30.0. Force mcp>=1.30.0,<2 through the existing overrides file, as already done for pyjwt. The CI scan never starts semgrep's MCP server. | 5 小时前 | |
fix(ci): migrate security scan from Safety to pip audit | 1 个月前 | |
fix(deps): override semgrep's mcp pin to clear GHSA-84m7-p3x7-pcfv (#2049) semgrep (through 1.181.0) pins mcp==1.29.0, which is affected by the unbounded Streamable HTTP session advisory fixed in 1.30.0. Force mcp>=1.30.0,<2 through the existing overrides file, as already done for pyjwt. The CI scan never starts semgrep's MCP server. | 5 小时前 | |
fix(ci): hash-pin every pip install for Scorecard Pinned-Dependencies Scorecard's Pinned-Dependencies check requires pip installs to be hash-verified, not just version-pinned - our existing pkg==X.Y.Z pins (and even pip install -r requirements-ci.txt, despite that file already carrying hashes) still scored a 4 because the pin/hash isn't visible on the command line itself. Adds .github/requirements/*.txt: hash-locked files generated via `uv pip compile --generate-hashes` for every pip target that isn't already requirements-ci.txt, covering standalone CI tooling (build, wheel, twine, uv, pip-audit, safety/bandit/semgrep/jq, pip/setuptools bootstrap) and the project's own local-source installs. The latter (`pip install -e ".[explorer]"`, `pip install -e .`) can't be hash-pinned directly since there's nothing to hash for a local source tree; split into `pip install --no-deps -e .` plus a separate hash-pinned install of the actual fetched dependencies instead. Also adds --require-hashes to every `-r requirements-ci.txt` install so hash verification is enforced explicitly rather than only implied by the file's own content. Simplifies the Dockerfile in the process: it now installs from the same pre-generated explorer-extra.txt (copied in at build time) instead of extracting constraints from requirements-ci.txt at build time, which also means setuptools gets its CVE-2025-47273 fix as a side effect of the hash-pinned install rather than a separate upgrade step. benchmark.yml: pip install -r benchmarks/requirements.txt is left unpinned - that directory doesn't exist in this repo, so there's nothing to generate hashes from. Pre-existing breakage, unrelated to this change. | 1 个月前 | |
Fix CurrencyNormalizer dropping K/M/B magnitude suffixes (#1854) * fix(normalize): preserve K/M/B magnitude suffixes in currency parsing * docs(normalize): add K/M/B magnitude suffix example * fix(normalize): reject malformed amounts and support bare leading decimals Addresses PR review feedback: the fallback parser inflated bare leading-decimal amounts like ".5" (parsed as 5 instead of 0.5) and silently accepted malformed numbers like "$1.2.3" (parsed as 1.2 instead of None). * fix(deps): bump urllib3 to 2.8.0 to resolve pip-audit CVEs * fix(deps): bump virtualenv to 21.9.0 to resolve pip-audit CVEs | 19 天前 | |
fix(ci): hash-pin every pip install for Scorecard Pinned-Dependencies Scorecard's Pinned-Dependencies check requires pip installs to be hash-verified, not just version-pinned - our existing pkg==X.Y.Z pins (and even pip install -r requirements-ci.txt, despite that file already carrying hashes) still scored a 4 because the pin/hash isn't visible on the command line itself. Adds .github/requirements/*.txt: hash-locked files generated via `uv pip compile --generate-hashes` for every pip target that isn't already requirements-ci.txt, covering standalone CI tooling (build, wheel, twine, uv, pip-audit, safety/bandit/semgrep/jq, pip/setuptools bootstrap) and the project's own local-source installs. The latter (`pip install -e ".[explorer]"`, `pip install -e .`) can't be hash-pinned directly since there's nothing to hash for a local source tree; split into `pip install --no-deps -e .` plus a separate hash-pinned install of the actual fetched dependencies instead. Also adds --require-hashes to every `-r requirements-ci.txt` install so hash verification is enforced explicitly rather than only implied by the file's own content. Simplifies the Dockerfile in the process: it now installs from the same pre-generated explorer-extra.txt (copied in at build time) instead of extracting constraints from requirements-ci.txt at build time, which also means setuptools gets its CVE-2025-47273 fix as a side effect of the hash-pinned install rather than a separate upgrade step. benchmark.yml: pip install -r benchmarks/requirements.txt is left unpinned - that directory doesn't exist in this repo, so there's nothing to generate hashes from. Pre-existing breakage, unrelated to this change. | 1 个月前 | |
fix(ci): hash-pin every pip install for Scorecard Pinned-Dependencies Scorecard's Pinned-Dependencies check requires pip installs to be hash-verified, not just version-pinned - our existing pkg==X.Y.Z pins (and even pip install -r requirements-ci.txt, despite that file already carrying hashes) still scored a 4 because the pin/hash isn't visible on the command line itself. Adds .github/requirements/*.txt: hash-locked files generated via `uv pip compile --generate-hashes` for every pip target that isn't already requirements-ci.txt, covering standalone CI tooling (build, wheel, twine, uv, pip-audit, safety/bandit/semgrep/jq, pip/setuptools bootstrap) and the project's own local-source installs. The latter (`pip install -e ".[explorer]"`, `pip install -e .`) can't be hash-pinned directly since there's nothing to hash for a local source tree; split into `pip install --no-deps -e .` plus a separate hash-pinned install of the actual fetched dependencies instead. Also adds --require-hashes to every `-r requirements-ci.txt` install so hash verification is enforced explicitly rather than only implied by the file's own content. Simplifies the Dockerfile in the process: it now installs from the same pre-generated explorer-extra.txt (copied in at build time) instead of extracting constraints from requirements-ci.txt at build time, which also means setuptools gets its CVE-2025-47273 fix as a side effect of the hash-pinned install rather than a separate upgrade step. benchmark.yml: pip install -r benchmarks/requirements.txt is left unpinned - that directory doesn't exist in this repo, so there's nothing to generate hashes from. Pre-existing breakage, unrelated to this change. | 1 个月前 |
CI tool requirements
Hash-pinned pip install targets for CI/release/Dockerfile steps that install
something other than the project's own audited requirements-ci.txt set.
These exist because OpenSSF Scorecard's Pinned-Dependencies check flags any
pip install in a workflow or Dockerfile that isn't hash-verified, and
requirements-ci.txt alone doesn't cover build/release/security tooling or
the project's own local-source install.
Each .txt was generated from the adjacent .in (or, for explorer-extra-py311.txt,
explorer-extra-py313.txt, and base-deps.txt, from pyproject.toml directly) with:
uv pip compile <input> --python-version 3.11 --python-platform linux \
--constraint requirements-ci.txt --generate-hashes -o <output>.txt
(--constraint requirements-ci.txt is omitted for bootstrap.txt,
build-tools.txt, uv-tool.txt, twine.txt, pip-audit.txt, and
security-scan-tools.txt, since those install standalone tooling with no
version relationship to the project's own dependency tree.)
security-scan-tools.txt is additionally compiled with
--override security-scan-tools-overrides.txt to force a patched pyjwt past
semgrep's pin, and is therefore installed with --no-deps in
security-scan.yml (see the overrides file for details).
Regenerate a file the same way after bumping a pinned version, and re-run it
whenever requirements-ci.txt changes if the file used --constraint (see
each file's own autogenerated header comment for its exact command).
| File | Used by | Installs |
|---|---|---|
bootstrap.txt |
security-scan.yml, benchmark.yml | pip, setuptools (upgrade before anything else) |
pep517-build.txt |
ci.yml, benchmark.yml, Dockerfile | exact [build-system] requires from pyproject.toml (setuptools, wheel) - installed with --no-build-isolation before any pip install -e . / pip install ., since --no-deps alone doesn't stop pip's PEP 517 build isolation from fetching those two unhashed |
explorer-extra-py311.txt |
ci.yml | semantica's base deps + the explorer extra, resolved for python 3.11 |
explorer-extra-py313.txt |
Dockerfile | the same, resolved for python 3.13 (the image's actual interpreter) |
pytest-tool.txt |
ci.yml | pytest, for the pre-all-extras deterministic test |
uv-tool.txt |
ci.yml | uv, to verify requirements-ci.txt is current |
build-tools.txt |
ci.yml, release.yml | build, wheel |
twine.txt |
release.yml | twine |
pip-audit.txt |
security-scan.yml | pip-audit |
security-scan-tools.txt |
security-scan.yml | bandit, semgrep, jq |
base-deps.txt |
benchmark.yml | semantica's base deps (no extras) |
benchmark-extra.txt |
benchmark.yml | the benchmark-only libs (neo4j, pdfplumber, etc.) |
explorer-extra-py31{1,3}.txt and base-deps.txt are large (they mirror
most of requirements-ci.txt) because semantica's dependencies list in
pyproject.toml isn't extras-gated - installing the package at all pulls
the full base set. That's expected, not a mistake.
explorer-extra-py311.txt and explorer-extra-py313.txt are not
interchangeable, and can't be collapsed into one file compiled for either
version: librosa's audioread dependency needs standard-aifc /
standard-sunau only under python_version >= "3.13" (Python 3.13 dropped
aifc/sunau from stdlib). A file resolved for 3.11 simply omits those
packages' hashes, so installing it with --require-hashes on a real 3.13
interpreter (the Dockerfile's base image) fails outright rather than
silently under-pinning. Any other file shared across a 3.11 and 3.13
consumer would need the same split if it hits a similar stdlib-removal
edge case - check for ERROR: In --require-hashes mode, all requirements must have their versions pinned on the other Python version before
assuming one --python-version covers every consumer.