我建立了一个有 65 个输入神经元和 4880 个输出神经元的模型。我的训练数据存储在两个大型文本文件中：“X_train.txt”包含几行，每行代表一个包含 65 个数字的列表，一个...
我建立了一个有 65 个输入神经元和 4880 个输出神经元的模型。我的训练数据存储在两个大型文本文件中：“X_train.txt”包含每行代表 65 个数字的列表，“Y_train.txt”包含每行代表一个索引号的行。我需要对“Y_train”执行独热编码，以在指定索引处创建一个包含 4880 个零和一个“1”的列表。
由于这些文件的大小，我想批量训练我的模型。如何使用 TensorFlow 在 Python 中有效地使用这些 txt 文件训练我的模型？
所以我尝试获取这些文件并尝试将这些文件转换为变量
with open(xPath, 'r') as file:
line = file.readline()
while line:
x_train.append(eval(line.strip()))
line = file.readline()
x_train = np.array(x_train)
但由于文件太大，需要花费太多时间，这不是我可以等待的主要问题，但主要问题是它使用了太多内存......
尝试使用此函数来解析每一行。然后将其映射到 tensorflow 数据集中。
def parse_x(line):
# Convert the line to a list of floats
values = tf.strings.to_number(tf.strings.split(line, ' '), tf.float32)
return values
def parse_y(line):
index = tf.strings.to_number(line, tf.int32)
one_hot = tf.one_hot(index, num_outputs)
return one_hot
我有一台 Macbook M3 Pro，并将所有内容从较旧的 Intel Pro 转移过来。我安装了 pyenv，但尝试安装任何版本的 Python 时都会出现构建错误。以下是一个例子：python-
我有一台 Macbook M3 Pro，并将所有内容从较旧的 Intel Pro 转移过来。我安装了 pyenv，但尝试安装任何版本的 Python 时都会出现构建错误。以下是示例：
python-build: use openssl@3 from homebrew
python-build: use readline from homebrew
Downloading Python-3.11.9.tar.xz...
-> https://www.python.org/ftp/python/3.11.9/Python-3.11.9.tar.xz
Installing Python-3.11.9...
python-build: use tcl-tk from homebrew
python-build: use readline from homebrew
python-build: use ncurses from homebrew
python-build: use zlib from xcode sdk
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/Users/ashtonsperry/.pyenv/versions/3.11.9/lib/python3.11/curses/__init__.py", line 13, in <module>
from _curses import *
ModuleNotFoundError: No module named '_curses'
WARNING: The Python curses extension was not compiled. Missing the ncurses lib?
Traceback (most recent call last):
File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'readline'
WARNING: The Python readline extension was not compiled. Missing the GNU readline lib?
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/Users/ashtonsperry/.pyenv/versions/3.11.9/lib/python3.11/ssl.py", line 100, in <module>
import _ssl # if we can't import it, let the error propagate
^^^^^^^^^^^
ModuleNotFoundError: No module named '_ssl'
ERROR: The Python ssl extension was not compiled. Missing the OpenSSL lib?
Please consult to the Wiki page to fix the problem.
https://github.com/pyenv/pyenv/wiki/Common-build-problems
BUILD FAILED (OS X 14.5 using python-build 20180424)
Inspect or clean up the working tree at /var/folders/p3/7nvxx8mx4x19125mj6fnjj0h0000gn/T/python-build.20240708161245.10785
Results logged to /var/folders/p3/7nvxx8mx4x19125mj6fnjj0h0000gn/T/python-build.20240708161245.10785.log
Last 10 log lines:
DYLD_LIBRARY_PATH=/var/folders/p3/7nvxx8mx4x19125mj6fnjj0h0000gn/T/python-build.20240708161245.10785/Python-3.11.9 ./python.exe -E -m ensurepip \
$ensurepip --root=/ ; \
fi
Looking in links: /var/folders/p3/7nvxx8mx4x19125mj6fnjj0h0000gn/T/tmp8_w0_bq0
Processing /var/folders/p3/7nvxx8mx4x19125mj6fnjj0h0000gn/T/tmp8_w0_bq0/setuptools-65.5.0-py3-none-any.whl
Processing /var/folders/p3/7nvxx8mx4x19125mj6fnjj0h0000gn/T/tmp8_w0_bq0/pip-24.0-py3-none-any.whl
Installing collected packages: setuptools, pip
WARNING: The scripts pip3 and pip3.11 are installed in '/Users/ashtonsperry/.pyenv/versions/3.11.9/bin' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed pip-24.0 setuptools-65.5.0
我知道这与 Apple 的新 M 芯片有关，我也知道我的 Brew 在 x86 上运行，而我的系统是 arm64。但我不知道该怎么做才能解决这个问题。我尝试过 Python 3.9.x、3.10.x 和 3.11.x，但都没有成功。这一切都始于我的 tensorflow 杀死了 Jupyter Notebooks 上的内核。