Skip to content

Sourcery refactored main branch#1

Open
sourcery-ai[bot] wants to merge 1 commit intomainfrom
sourcery/main
Open

Sourcery refactored main branch#1
sourcery-ai[bot] wants to merge 1 commit intomainfrom
sourcery/main

Conversation

@sourcery-ai
Copy link
Copy Markdown

@sourcery-ai sourcery-ai bot commented Dec 24, 2023

Branch main refactored by Sourcery.

If you're happy with these changes, merge this Pull Request using the Squash and merge strategy.

See our documentation here.

Run Sourcery locally

Reduce the feedback loop during development by using the Sourcery editor plugin:

Review changes via command line

To manually merge these changes, make sure you're on the main branch, then run:

git fetch origin sourcery/main
git merge --ff-only FETCH_HEAD
git reset HEAD^

Help us improve this pull request!

@sourcery-ai sourcery-ai bot requested a review from Abaso007 December 24, 2023 07:09
@cr-gpt
Copy link
Copy Markdown

cr-gpt bot commented Dec 24, 2023

Seems you are using me but didn't get OPENAI_API_KEY seted in Variables/Secrets for this repo. you could follow readme for more information

Comment on lines -48 to +54
num_parts = 0

for filename in os.listdir(dir_model):
if filename.startswith("pytorch_model-"):
num_parts += 1

num_parts = sum(
1
for filename in os.listdir(dir_model)
if filename.startswith("pytorch_model-")
)
if num_parts > 0:
print("gguf: found " + str(num_parts) + " model parts")
print(f"gguf: found {str(num_parts)} model parts")
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function count_model_parts refactored with the following changes:

Comment on lines -139 to +141
num_parts = 0
for filename in os.listdir(dir_model):
if filename.endswith(prefix):
num_parts += 1

return num_parts
return sum(
1 for filename in os.listdir(dir_model) if filename.endswith(prefix)
)
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function Model.count_model_parts refactored with the following changes:

Comment on lines -789 to +787
self.gguf_writer.add_rope_dimension_count(hidden_size // head_count)
self.gguf_writer.add_head_count(head_count)
self.gguf_writer.add_rope_dimension_count(hidden_size // head_count_kv)
self.gguf_writer.add_head_count(head_count_kv)
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function PersimmonModel.set_gguf_parameters refactored with the following changes:

Comment on lines -168 to +170
num_parts = 0
for filename in os.listdir(dir_model):
if filename.endswith(prefix):
num_parts += 1

return num_parts
return sum(
1 for filename in os.listdir(dir_model) if filename.endswith(prefix)
)
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function Model.count_model_parts refactored with the following changes:

Comment on lines -217 to +214
if arch == "RWForCausalLM" or arch == "LlamaForCausalLM":
if arch in ["RWForCausalLM", "LlamaForCausalLM"]:
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function Model._get_model_architecture refactored with the following changes:

Comment on lines -334 to +338
added_tokens = dict(
(item['content'], item['id'])
added_tokens = {
item['content']: item['id']
for item in tokenizer_json.get('added_tokens', [])
# Added tokens here can be duplicates of the main vocabulary.
if item['content'] not in self.bpe_tokenizer )
if item['content'] not in self.bpe_tokenizer
}
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function BpeVocab.__init__ refactored with the following changes:

Comment on lines +363 to -364
score = -1000.0
for text in self.added_tokens_list:
score = -1000.0
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function BpeVocab.added_tokens refactored with the following changes:

Comment on lines +425 to -426
score = -1000.0
for text in self.added_tokens_list:
score = -1000.0
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function SentencePieceVocab.added_tokens refactored with the following changes:


def merge_multifile_models(models_plus: list[ModelPlus]) -> ModelPlus:
formats = set(mp.format for mp in models_plus)
formats = {mp.format for mp in models_plus}
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function merge_multifile_models refactored with the following changes:

Comment on lines -625 to +631
return LazyTensor(load, lazy_tensor.shape, lazy_tensor.data_type, f'permute({n_head}, {n_head_kv}) ' + lazy_tensor.description)

return LazyTensor(
load,
lazy_tensor.shape,
lazy_tensor.data_type,
f'permute({n_head}, {n_head_kv}) {lazy_tensor.description}',
)
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function permute_lazy refactored with the following changes:

Comment on lines +636 to +644

s = lazy_tensor.shape.copy()
s[0] = s[0] // 3
return LazyTensor(load, s, lazy_tensor.data_type, f'permute({n_head}, {n_head_kv}) ' + lazy_tensor.description)
return LazyTensor(
load,
s,
lazy_tensor.data_type,
f'permute({n_head}, {n_head_kv}) {lazy_tensor.description}',
)
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function permute_part_lazy refactored with the following changes:

Comment on lines +649 to +654

s = lazy_tensor.shape.copy()
s[0] = s[0] // 3
return LazyTensor(load, s, lazy_tensor.data_type, 'part ' + lazy_tensor.description)
return LazyTensor(
load, s, lazy_tensor.data_type, f'part {lazy_tensor.description}'
)
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function part_lazy refactored with the following changes:

Comment on lines -817 to +846
if params.n_vocab != vocab.vocab_size:
assert isinstance(vocab, BpeVocab) or isinstance(vocab, SentencePieceVocab)
if params.n_vocab == vocab.vocab_size_base:
print("Ignoring added_tokens.json since model matches vocab size without it.")
vocab.added_tokens_list = []
vocab.vocab_size = vocab.vocab_size_base
return
msg = f"Vocab size mismatch (model has {params.n_vocab}, but {vocab.fname_tokenizer}"
if vocab.fname_added_tokens is not None:
msg += f" combined with {vocab.fname_added_tokens}"
msg += f" has {vocab.vocab_size})."
if vocab.vocab_size < params.n_vocab < vocab.vocab_size + 20 and vocab.fname_added_tokens is None:
msg += f" Most likely you are missing added_tokens.json (should be in {vocab.fname_tokenizer.parent})."
raise Exception(msg)
if params.n_vocab == vocab.vocab_size:
return
assert isinstance(vocab, (BpeVocab, SentencePieceVocab))
if params.n_vocab == vocab.vocab_size_base:
print("Ignoring added_tokens.json since model matches vocab size without it.")
vocab.added_tokens_list = []
vocab.vocab_size = vocab.vocab_size_base
return
msg = f"Vocab size mismatch (model has {params.n_vocab}, but {vocab.fname_tokenizer}"
if vocab.fname_added_tokens is not None:
msg += f" combined with {vocab.fname_added_tokens}"
msg += f" has {vocab.vocab_size})."
if vocab.vocab_size < params.n_vocab < vocab.vocab_size + 20 and vocab.fname_added_tokens is None:
msg += f" Most likely you are missing added_tokens.json (should be in {vocab.fname_tokenizer.parent})."
raise Exception(msg)
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function check_vocab_size refactored with the following changes:

Comment on lines -940 to +956
if not isinstance(dt, QuantizedDataType):
return arr
return dt.quantize(arr)
return arr if not isinstance(dt, QuantizedDataType) else dt.quantize(arr)
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function OutputFile.maybe_do_quantize refactored with the following changes:

Comment on lines -980 to +996
wq_type = model[gguf.TENSOR_NAMES[gguf.MODEL_TENSOR.ATTN_Q].format(bid=0)+".weight"].data_type
wq_type = model[
f"{gguf.TENSOR_NAMES[gguf.MODEL_TENSOR.ATTN_Q].format(bid=0)}.weight"
].data_type
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function pick_output_type refactored with the following changes:

Comment on lines -138 to +150
self.past = struct.unpack('<i', bytes(data[offset:offset + 4]))[0]; offset += 4
self.lbfgs_m = struct.unpack('<i', bytes(data[offset:offset + 4]))[0]; offset += 4
self.nx = struct.unpack('N', bytes(data[offset:offset + 8]))[0]; offset += 8
self.iter = struct.unpack('<i', bytes(data[offset:offset + 4]))[0]; offset += 4
self.just_initialized = bool(struct.unpack('<i', bytes(data[offset:offset + 4]))[0]); offset += 4
self.past = struct.unpack('<i', bytes(data[offset:offset + 4]))[0]
offset += 4
self.lbfgs_m = struct.unpack('<i', bytes(data[offset:offset + 4]))[0]
offset += 4
self.nx = struct.unpack('N', bytes(data[offset:offset + 8]))[0]
offset += 8
self.iter = struct.unpack('<i', bytes(data[offset:offset + 4]))[0]
offset += 4
self.just_initialized = bool(struct.unpack('<i', bytes(data[offset:offset + 4]))[0])
offset += 4
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function OptimizationContext.load refactored with the following changes:

Comment on lines -433 to +470
magic = bytes(reversed(data[offset:offset + 4])); offset += 4
magic = bytes(reversed(data[offset:offset + 4]))
offset += 4
if magic != b'ggcl':
raise ValueError(f"File header magic indicates, that this is no finetune-lora checkpoint file. Expected 'ggcl', Got '{str(magic)}'")
raise ValueError(
f"File header magic indicates, that this is no finetune-lora checkpoint file. Expected 'ggcl', Got '{magic}'"
)

self.version = struct.unpack('<I', bytes(data[offset:offset + 4]))[0]; offset += 4
self.version = struct.unpack('<I', bytes(data[offset:offset + 4]))[0]
offset += 4
if self.version != 0:
raise ValueError('Invalid version of checkpoint file')

self.train_its = struct.unpack('<I', bytes(data[offset:offset + 4]))[0]; offset += 4
self.train_samples = struct.unpack('<I', bytes(data[offset:offset + 4]))[0]; offset += 4
self.train_tokens = struct.unpack('<I', bytes(data[offset:offset + 4]))[0]; offset += 4
self.train_its = struct.unpack('<I', bytes(data[offset:offset + 4]))[0]
offset += 4
self.train_samples = struct.unpack('<I', bytes(data[offset:offset + 4]))[0]
offset += 4
self.train_tokens = struct.unpack('<I', bytes(data[offset:offset + 4]))[0]
offset += 4
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function LoraCheckpoint.load refactored with the following changes:

filename = os.fsdecode(listing)
if filename.endswith(".txt"):
file = open("./examples/jeopardy/results/" + filename, "rt")
file = open(f"./examples/jeopardy/results/{filename}", "rt")
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function calculatecorrect refactored with the following changes:

Comment on lines -19 to +36
if name in (
if name in {
"logit_scale",
"text_model.embeddings.position_ids",
"vision_model.embeddings.position_ids",
):
}:
return True

if has_llava and name in ["visual_projection.weight", "vision_model.post_layernorm.weight", "vision_model.post_layernorm.bias"]:
if has_llava and name in {
"visual_projection.weight",
"vision_model.post_layernorm.weight",
"vision_model.post_layernorm.bias",
}:
return True

if name.startswith("v") and not has_vision:
return True

if name.startswith("t") and not has_text:
return True

return False
return bool(name.startswith("t") and not has_text)
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function should_skip_tensor refactored with the following changes:

Comment on lines -100 to +105
with open(dir_model + "/vocab.json", "r", encoding="utf-8") as f:
with open(f"{dir_model}/vocab.json", "r", encoding="utf-8") as f:
vocab = json.load(f)
tokens = [key for key in vocab]
tokens = list(vocab)

with open(dir_model + "/config.json", "r", encoding="utf-8") as f:
with open(f"{dir_model}/config.json", "r", encoding="utf-8") as f:
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lines 100-250 refactored with the following changes:

Comment on lines -26 to +28
# BakLLaVA models contain CLIP tensors in it
clip_tensors = [k for k, v in checkpoint.items() if k.startswith("model.vision_tower")]
if len(clip_tensors) > 0:
if clip_tensors := [
k for k, v in checkpoint.items() if k.startswith("model.vision_tower")
]:
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lines 26-28 refactored with the following changes:

This removes the following comments ( why? ):

# BakLLaVA models contain CLIP tensors in it

if json[key] == None:
return False
return True
return json[key] is not None
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function is_present refactored with the following changes:

postData["prompt"] = body["prompt"]
postData = {
"prompt": convert_chat(body["messages"]) if chat else body["prompt"]
}
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function make_postData refactored with the following changes:

self.nbytes = 0
else:
self.nbytes = int(np.product(self.ne)) * 4
self.nbytes = 0 if len(self.ne) == 0 else int(np.product(self.ne)) * 4
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function Tensor.__init__ refactored with the following changes:

Comment on lines -74 to +82
nd = struct.unpack('<I', bytes(data[offset:offset + 4]))[0]; offset += 4
namelen = struct.unpack('<I', bytes(data[offset:offset + 4]))[0]; offset += 4
dtype = struct.unpack('<I', bytes(data[offset:offset + 4]))[0]; offset += 4
nd = struct.unpack('<I', bytes(data[offset:offset + 4]))[0]
offset += 4
namelen = struct.unpack('<I', bytes(data[offset:offset + 4]))[0]
offset += 4
dtype = struct.unpack('<I', bytes(data[offset:offset + 4]))[0]
offset += 4

assert(nd == len(self.ne))
ne = []
for d in range(nd):
n = struct.unpack('<I', bytes(data[offset:offset + 4]))[0]; offset += 4
for _ in range(nd):
n = struct.unpack('<I', bytes(data[offset:offset + 4]))[0]
offset += 4
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function Tensor.load refactored with the following changes:

Comment on lines -31 to +49
for i in range(layer):
for _ in range(layer):
values += [0.0]
c = np.array(values, dtype=float)
c = matrix(c)

# Setting capacity and neuron count per batch
CAP = capacity
CAP = int(CAP / batch)
neuron = int(neuron / batch)
coeff = []
h = []

# Constraint 1: Total neuron activation constraint
lst = []
for i in range(neuron * layer):
lst.append(1)
for i in range(layer):
lst.append(0)
coeff.append(lst)
h.append(CAP)

CAP //= batch
neuron //= batch
lst = [1 for _ in range(neuron * layer)]
lst.extend(0 for _ in range(layer))
coeff = [lst]
h = [CAP]
# Constraint 2: Threshold constraint for GPU split per layer
for i in range(layer):
lst = [0] * (neuron * layer + layer)
for j in range(neuron):
lst[i * neuron + j] = -1
lst[neuron * layer + i] = int(threshold / batch)
lst[neuron * layer + i] = threshold // batch
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function solve_gpu_split refactored with the following changes:

This removes the following comments ( why? ):

# Constraint 1: Total neuron activation constraint

Comment on lines 14 to 18
while True:
n = f.readinto(mv)
if not n:
break
file_hash.update(mv[:n])
if n := f.readinto(mv):
file_hash.update(mv[:n])

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function sha256sum refactored with the following changes:

Comment on lines -57 to +58
if file_hash == hash_value:
valid_checksum = "V"
file_missing = ""
else:
valid_checksum = ""
file_missing = ""
valid_checksum = "V" if file_hash == hash_value else ""
file_missing = ""
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lines 57-62 refactored with the following changes:

k = text.replace('\n', '\\n')
k = k.replace('\t', '\\t')
k = '"' + k + '"'
k = f'"{k}"'
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lines 59-73 refactored with the following changes:

dir_tokenizer = args.dir_tokenizer

tokenizer = SentencePieceProcessor(dir_tokenizer + '/tokenizer.model')
tokenizer = SentencePieceProcessor(f'{dir_tokenizer}/tokenizer.model')
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lines 16-83 refactored with the following changes:

This removes the following comments ( why? ):

# '_Hello'
# 'Hello'
# '_'
# ' Hello'
# ' Hello  Hello'
# 'Hello Hello'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants