Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: Tiiny-AI/PowerInfer
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: main
Choose a base ref
...
head repository: Tiiny-AI/PowerInfer
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: fix/vram-budget-inaccuracy
Choose a head ref
Checking mergeability… Don’t worry, you can still create the pull request.
  • 12 commits
  • 3 files changed
  • 1 contributor

Commits on Apr 10, 2024

  1. fix: calculated weight size per neuron with wrong dim

    @hodlen
    hodlen committed Apr 10, 2024
    Configuration menu
    Copy the full SHA
    250dcac View commit details
    Browse the repository at this point in the history

  2. debug model layers access - SHOULD BE REVERTED

    @hodlen
    hodlen committed Apr 10, 2024
    Configuration menu
    Copy the full SHA
    53bb16e View commit details
    Browse the repository at this point in the history

  3. wip: stop estimating and reserving kv cache

    @hodlen
    hodlen committed Apr 10, 2024
    Configuration menu
    Copy the full SHA
    b1d435e View commit details
    Browse the repository at this point in the history

Commits on Apr 11, 2024

  1. wip alloc tensor weights after first graph built

    @hodlen
    hodlen committed Apr 11, 2024
    Configuration menu
    Copy the full SHA
    10a006b View commit details
    Browse the repository at this point in the history

Commits on Apr 12, 2024

  1. wip

    @hodlen
    hodlen committed Apr 12, 2024
    Configuration menu
    Copy the full SHA
    77bcb88 View commit details
    Browse the repository at this point in the history

Commits on Apr 15, 2024

  1. wip

    @hodlen
    hodlen committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    16aaf7d View commit details
    Browse the repository at this point in the history

Commits on Apr 25, 2024

  1. fix: no gpu index in graph when estimating context vram

    @hodlen
    hodlen committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    a8cbbb3 View commit details
    Browse the repository at this point in the history

  2. cleanup

    @hodlen
    hodlen committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    77b38fe View commit details
    Browse the repository at this point in the history

Commits on Apr 26, 2024

  1. wip

    @hodlen
    hodlen committed Apr 26, 2024
    Configuration menu
    Copy the full SHA
    d7fa936 View commit details
    Browse the repository at this point in the history

  2. revert pre-allocating graph for vram estimation

    @hodlen
    hodlen committed Apr 26, 2024
    Configuration menu
    Copy the full SHA
    ed393c8 View commit details
    Browse the repository at this point in the history

Commits on Apr 27, 2024

  1. wip: create full gpu ctx to compute buffer size

    @hodlen
    hodlen committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    e1802b9 View commit details
    Browse the repository at this point in the history

  2. wip

    @hodlen
    hodlen committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    a915f9a View commit details
    Browse the repository at this point in the history
Loading