[linux] huggingface transformers 如何下载模型至本地 & git lfs install 报错

bert-base-uncased at main

一、官方给出的命令：

# Make sure you have git-lfs installed (https://git-lfs.com)
git lfs install
git clone https://huggingface.co/bert-base-uncased



# if you want to clone without large files – just their pointers
# prepend your git clone with the following env var:
GIT_LFS_SKIP_SMUDGE=1

但是 git lfs install 的时候报错。

于是查了一下。。。。安装lfs是要这样： 

二、安装 lfs

You can't directly use

git lfs install

Instead of that, you can use these commands to download and install (you have to download it before installing).

# for linux
# 如果报错，则用 sudo apt-get update
sudo apt-get install git-lfs
git-lfs install
# for mac
brew install git-lfs
#用git-lfs install检查是否正确安装
git-lfs install

网络不好mac报错：重新安了一下brew install git-lfs。。

三、 再用官方的命令下载。

如果失败，则。。。可能是网络太差。。。。

最终，我还是给 from_pretrained 套了循环，来解决的。。。。。。。

如果不套循环，会报错 " requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))" 。。。。。。。。。。。。。。。。。。。。。文章来源地址https://www.toymoban.com/news/detail-688167.html

class Trainer(object):
    def __init__(self, args, train_dataset=None, dev_dataset=None, test_dataset=None):
        self.args = args
        self.train_dataset = train_dataset
        self.dev_dataset = dev_dataset
        self.test_dataset = test_dataset

        self.intent_label_lst = get_intent_labels(args)
        self.slot_label_lst = get_slot_labels(args)
        # Use cross entropy ignore index as padding label id so that only real label ids contribute to the loss later
        self.pad_token_label_id = args.ignore_index

        self.config_class, self.model_class, _ = MODEL_CLASSES[args.model_type]
        #self.config = self.config_class.from_pretrained(args.model_name_or_path, finetuning_task=args.task, output_hidden_states=args.output_hidden_states)
        self.config = self.config_class.from_pretrained(args.model_name_or_path, finetuning_task=args.task)
        ################## [O.O]这是一个循环，解决下不下来模型的问题 #################
        nb_tries = 20
        while nb_tries>0:
            nb_tries -= 1
            try:
                self.model = self.model_class.from_pretrained(args.model_name_or_path,
                                                      config=self.config,
                                                      args=args,
                                                      intent_label_lst=self.intent_label_lst,
                                                      slot_label_lst=self.slot_label_lst)
                break
            except:
                time.sleep(0.1)
        #########################################################################
        # GPU or CPU
        self.device = "cuda" if torch.cuda.is_available() else "cpu"
        # self.device = "cuda" if torch.cuda.is_available() and not args.no_cuda else "cpu"
        self.model.to(self.device)

到了这里，关于[linux] huggingface transformers 如何下载模型至本地 & git lfs install 报错的文章就介绍完了。如果您还想了解更多内容，请在右上角搜索TOY模板网以前的文章或继续浏览下面的相关文章，希望大家以后多多支持TOY模板网！