
One Section
Add a review FollowOverview
-
Founded Date December 27, 2009
-
Sectors Office
-
Posted Jobs 0
-
Viewed 7
Company Description
Open-R1: a Fully Open Reproduction Of DeepSeek-R1
Hey there! This blog post is an introduction to the job, not a claim that we’ve reproduced R1 yet. We’re integrating in the open, so as quickly as we have evaluation numbers, we’ll share them. You can follow our development on Hugging Face and GitHub.
True, however it looks like there’s nothing to be examined since right now. I presume the ultimate goal is to train a new reasoning design and then use the exact same evaluation metrics as o1 and the DeepSeek-R1.
Well, there should be at least some peace of mind check and validation to guarantee the design was trained correctly.
Oh yes, if you are discussing the examination variety of deepseek’s design it’s coming soon!
As discussed in the article there is no design called Open-R1 to check at all … not yet anyway. This is a blog site describing that Hugging face will take the R1 Deepseek design, exercise how it was constructed as laid out in the paper and from what they released, and after that reproduce that process.
in reality this is basically how science works … A creates a strategy, discovery or innovation and it is evaluated by B, C and D to see if it is reproduceable. Thats been the cornerstone of research now for a couple of centuries.
This blog site is not stating they have actually already done so … Its a blog site laying out an intent to start training a design like R1 and calling it Open-R1.
Also DeepSeek-R1 was only released last week, and even in their paper they described the calculate hours required. While those are low calculate hours for a SOTA model this does not indicate you can train said design in a week. I ‘d personally enjoy to be able to train a transformer model in a week, but we may need to wait a while for that level of compute technology.
So there are no criteria for a design that has not been constructed yet right? As described in the blog, and once again in reply to your concern.
However fear not, there is a GitHub Repo already and contributors (hell I may join myself), some prelim work done, and a master plan. A good starting position.
n
@edbeeching
has assessed the launched designs already
( src: https://x.com/edwardbeeching/status/1884273209136275742)
R1 simply trained on o1 outputs, so jointly …/ s. This is what the brand-new AI czars are stating
Hi! This blog post is an intro to the project, not a claim that we have actually recreated R1 yet. We will completely share the missing out on piece when we have them, you can anticipate the models and datasets to be upload in this Hugging Face org and the code to be in this GitHub repo
That’s good and important to understand this significant buzz that does not have technical comprehension and explanation. Science is about recreation, and if they claim to be open, let them fullfill the open part.
Please do publish the training expense.
We will!
Excalidraw Hi n
@bojan2501
thanks, we will undoubtedly be striving to make certain this training dish can work for small language designs on customer hardware considering that not everybody has a cluster of H100s in the house:-RRB- The tool we used for the images was Excalidraw! https://excalidraw.com
looking forward to it! WTF are your speaking about?
must be a joke
It’s truly cool to see how the whole open source community comes together!
Ops …
5.5 M is number press reporter in the deepseekv3 tech report (simply the training, not the experiment afaik), for R1 tough to estimate tbh but much less than 5.5 M imo
Historically, they have never released code or datasets of their LLM training, so I wouldn’t anticipate this time to be various. If they would release it that would be incredible of course!
Yes naturally!
So generally you’re asking to replace existing censorship with another flavour of censorship?
The code for the designs are inside the design repositories, e.g. for V3: https://huggingface.co/deepseek-ai/DeepSeek-V3/blob/main/modeling_deepseek.py
Hello Team, I’m Ray Bernard, the author and creator of EQUATOR. My research study group will be dealing with a paper concentrated on replicating particular elements of DeepSeek R1. Our objective is to recreate the cold start and offer your group with a dataset that consists of COT and other methods to support these efforts. We like to contribute our work to help. Please let me know if you find this beneficial. Best, Ray Bernard https://www.facebook.com/groups/1186310571520299/
Where is the assessment numbers? without it you can’t call it reproduction.
8 replies
True, but it looks like there’s absolutely nothing to be assessed as of today. I assume the supreme goal is to train a brand-new reasoning model and after that utilize the very same examination metrics as o1 and the DeepSeek-R1.
That’s quite intriguing, I was asking myself why the questions the author exposed here are not being asked by others? I believe the work they have actually done is remarkable but at the exact same time I wonder why they wouldn’t put these missing out on pieces on if they are supposed to be totally open.
Why even without reproduction and comprehension of the development they could affect so much the market in this method?
4 replies
Hi! This blog post is an intro to the project, not a claim that we’ve recreated R1 yet. We will totally share the missing piece when we have them, you can expect the models and datasets to be upload in this Hugging Face org and the code to be in this GitHub repo
Interesting read, and it is excellent that we see more effort into this instructions: more optimization and less brute force.
Also question what tool did the author use for producing step diagram.
2 replies
Excalidraw I’m so delighted that initiative like this already exist, I’m gon na attempt to contribute:-RRB- 1 reply
looking forward to it! So racist articel
2 replies
WTF are your discussing?
Awesome to have this open recreation began!
For Step # 1 check out https://github.com/open-thoughts/open-thoughts!
https://x.com/ryanmart3n/status/1884284101265612856
Let’s do this thing!
1 reply
It’s truly cool to see how the whole open source community comes together!
Does anybody know the real training expense of r1? I can’t find it in the paper or the announcement post. Is the 6M expense reported by media simply the number drawn from v3’s training cost?
2 replies
Ops …
Has anybody asked the DeepSeek group to release their training data and code, or a minimum of share them independently with an independent duplication task like this? Have they rejected such a request?
A loyal replication depends upon using the exact same dataset and hyperparameters. Otherwise, any significant disparities with the published benchmarks would be difficult to pin down-whether due to training data distinctions or the replication method itself.
1 reply
Historically, they have actually never released code or datasets of their LLM training, so I would not anticipate this time to be various. If they would release it that would be fantastic naturally!
In the meantime we need to make best guess price quotes and see if we can get there ourselves.
You offer great replication process of Deepseek thinking training. I will try something comparable to it.
This is actually good information, can we fine tune with particular usage case when code is released?
1 reply
Yes obviously!
Please consider eliminating biased, polluted or unaligned training data and make an effort to eliminate copyrighted works from the crawl from intake. This will make the design more usable. If you recycled anthropic curation checks, this may also assist, get rid of obviouslybiased data will likely include a great deal of value. We do not want another tainted, unaligned open source design, right? And no corporate would ever utilize deepseek or a design that reuses it, right?
We appreciate your work for the advantage of mankind, we hope.
Miike C from NJ
1 reply
So basically you’re asking to replace existing censorship with another flavour of censorship?
Can’t wait! Hopefully the model will be uncensored however whatever you can do is alright! Love seeing open source building itself up. I’m not clever enough to actually help however I can contribute ethical lol
Hello guys, I am even just looking for code for DeepSeek-V2, in order to fully comprehend multi-head hidden attention. You do not seem to have code in Hugging Face even for that. Or am I missing something? Don’t see anything in src/transformers/models. MLA is not properly explained in their paper, so it would be essential to have code for this.