Skip to content

Add MMAP support #50

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
PicoCreator opened this issue May 5, 2023 · 2 comments
Open

Add MMAP support #50

PicoCreator opened this issue May 5, 2023 · 2 comments

Comments

@PicoCreator
Copy link
Contributor

Porting over the https://github.com/ggerganov/llama.cpp mmap support, would reduce the minimum RAM requirement for the model.

I know, you know, haha - just putting it here for me to keep track of (if i come around to picking it up, to optimize the RWKV-cpp-node bindings)
LoganDark added a commit to LoganDark/rwkv.cpp that referenced this issue May 24, 2023
@LoganDark
Move graph building into its own function
f422a3a 
step towards RWKV#50 and loading models from memory among other things
@LoganDark LoganDark mentioned this issue May 24, 2023
Merged
saharNooby pushed a commit that referenced this issue May 26, 2023
@LoganDark
Move graph building into its own function (#69)
3ca9c7f 
step towards #50 and loading models from memory among other things
@Ar57m
Copy link

Ar57m commented Sep 25, 2023

any updates?

@saharNooby
Copy link
Collaborator

@Ar57m Not a the moment. mmap support is in my backlog, but I have no estimates.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@saharNooby @PicoCreator @Ar57m