我正在开发一个神经网络，根据包含 1,200,000 个棋局的数据集来预测棋局。尽管数据集很大，但我的模型并没有适当地学习，无法达到峰值准确率。
我正在开发一个神经网络，根据包含 1,200,000 个棋局位置的数据集来预测棋局。尽管数据集很大，但我的模型并没有正确学习，峰值准确率仅为 0.0104。模型的输出与预期的 128 大小的独热编码数组不一致，这些数组代表有效的棋局。我已经验证了我的训练数据和输入张量，但神经网络的性能仍然不是最优的。我怀疑我的神经网络架构或训练过程可能有问题。
数据和模型详细信息输入张量：我使用 8x8x12 3D 张量来表示棋盘的状态，由 0 和 1 进行独热编码。输出张量：我使用大小为 128 的一维数组，也是独热编码，其中两个“1”表示从一个位置移动到另一个位置。数据集和预处理对于训练数据集，我使用了 20,000 场游戏 Kaggle CSV 数据集。我对每场比赛进行迭代，为每个连续的移动序列生成张量训练和结果数据，最终产生 1,200,000 个数据对移动点。我已经使用测试用例验证了我的训练数据，确保其正确性。
问题陈述尽管做了这些准备，但我的神经网络在训练过程中的准确率非常低。虽然我预计直接预测国际象棋动作会遇到一些挑战，但我主要的目标是让神经网络从数据集中学习基本的国际象棋规则。我的计划是使用后处理来过滤掉最可能的有效动作。然而，该模型的表现远低于预期。
问题模型架构：我的神经网络架构是否存在明显问题，可能导致性能如此糟糕？训练过程：我训练模型的方式是否存在问题，可以解释这些结果？改进建议：有什么建议可以改进模型的性能，至少可以学习基本的国际象棋规则并产生有效的移动输出？
#model class part of code, and dataset fetch game part
class ChessModel:
def __init__(self):
self.model = self.create_model()
def create_model(self):
model = tf.keras.models.Sequential([
# Convolutional Block 1
tf.keras.layers.Conv2D(64, (3, 3), activation='relu', input_shape=(8, 8, 12)),
tf.keras.layers.BatchNormalization(),
# Convolutional Block 2
tf.keras.layers.Conv2D(128, (3, 3), activation='relu'),
tf.keras.layers.BatchNormalization(),
# Convolutional Block 3
tf.keras.layers.Conv2D(256, (3, 3), activation='relu'),
tf.keras.layers.BatchNormalization(),
# Global Pooling Layer
tf.keras.layers.GlobalAveragePooling2D(),
# Dense Block
tf.keras.layers.Dense(1024, activation='relu'),
tf.keras.layers.Dropout(0.2),
tf.keras.layers.Dense(512, activation='relu'),
tf.keras.layers.Dropout(0.2),
tf.keras.layers.Dense(128, activation='softmax')
])
model.summary()
model.compile(optimizer='adam',
loss='categorical_crossentropy',
metrics=['accuracy'])
return model
def train_model(self, X_train, y_train, X_val, y_val):
self.model.fit(X_train, y_train, epochs=10, batch_size=128, validation_data=(X_val, y_val))
def save_model(self, model_name):
model_dir = r'C:\Users\schul\Desktop\Chess Engine Python\Neural_Net_Models'
if not os.path.exists(model_dir):
os.makedirs(model_dir)
model_path = os.path.join(model_dir, '{}.keras'.format(model_name))
try:
self.model.save(model_path)
print(f"Model saved successfully at {model_path}")
except Exception as e:
print(f"An error occurred while saving the model: {e}")
def process_games(self):
all_states = []
for index, row in self.games_df.iterrows():
moves_san = row['moves']
board = chess.Board()
move_list = moves_san.split()
uci_moves = []
for move_san in move_list:
try:
move = board.parse_san(move_san)
uci_move = move.uci()
uci_moves.append(uci_move)
board.push(move)
except ValueError:
continue
uci_moves_str = " ".join(uci_moves)
states = self.split_conversion_one_hot_127_array(uci_moves_str)
if states is not None: # Check states not none by prs.
all_states.extend(states)
self.all_current_states = []
self.all_next_states = []
for state in all_states:
if state is not None:
current_state, next_state = state
self.all_current_states.append(current_state)
self.all_next_states.append(next_state)
self.all_current_states = np.array(self.all_current_states)
self.all_next_states = np.array(self.all_next_states)
def split_data(self):
X_train, X_val, y_train, y_val = train_test_split(self.all_current_states, self.all_next_states, test_size=0.2, random_state=42)
print(len(self.all_current_states))
print(len(self.all_next_states))
print("x train e.g. length sizes: ")
print(len(X_train))
print(len(X_val))
print(len(y_train))
print(len(y_val))
return X_train, y_train, X_val, y_val
我对 AWS 还比较陌生，所以任何帮助都非常感谢！我已经设置了一个 Lmabda fn，当用户添加到 Cognito 中时，它会将用户添加到我的 RDS 数据库中。我正在尝试直接通过 Lam 运行测试...
我对 AWS 还比较陌生，所以任何帮助都非常感谢！我已经设置了一个 Lmabda fn，当用户被添加到 Cognito 中时，它会将用户添加到我的 RDS 数据库中。我正在尝试直接通过 Lambda 控制台页面运行测试。
我在运行测试时不断收到此错误：
Test Event Name
basicTest
Response
{
"statusCode": 500,
"body": "{\"message\":\"Failed to create user\",\"error\":\"no pg_hba.conf entry for host \\\"10.0.2.14\\\", user \\\"postgres\\\", database \\\"creditcuesdb\\\", no encryption\"}"
}
Function Logs
START RequestId: 10b5a33e-6fa2-4ca9-a0b2-7e0c0eee1b2a Version: $LATEST
2024-07-25T06:44:51.478Z 10b5a33e-6fa2-4ca9-a0b2-7e0c0eee1b2a INFO Event: {
"request": {
"userAttributes": {
"name": "John Doe",
"birthdate": "1990-01-01",
"email": "[email protected]",
"user_id": "12345"
}
}
}
2024-07-25T06:44:51.478Z 10b5a33e-6fa2-4ca9-a0b2-7e0c0eee1b2a INFO Connecting to the database with the following configuration:
2024-07-25T06:44:51.478Z 10b5a33e-6fa2-4ca9-a0b2-7e0c0eee1b2a INFO Host: cue-prod-v1-mydbinstance-vjg6ptc3cfhh.cexk8pzzn0qz.us-west-1.rds.amazonaws.com
2024-07-25T06:44:51.478Z 10b5a33e-6fa2-4ca9-a0b2-7e0c0eee1b2a INFO User: postgres
2024-07-25T06:44:51.478Z 10b5a33e-6fa2-4ca9-a0b2-7e0c0eee1b2a INFO Database: creditcuesdb
2024-07-25T06:44:51.478Z 10b5a33e-6fa2-4ca9-a0b2-7e0c0eee1b2a INFO Port: 5432
2024-07-25T06:44:51.600Z 10b5a33e-6fa2-4ca9-a0b2-7e0c0eee1b2a ERROR Error adding user to the database: error: no pg_hba.conf entry for host "10.0.2.14", user "postgres", database "creditcuesdb", no encryption
at Parser.parseErrorMessage (/var/task/node_modules/pg-protocol/dist/parser.js:283:98)
at Parser.handlePacket (/var/task/node_modules/pg-protocol/dist/parser.js:122:29)
at Parser.parse (/var/task/node_modules/pg-protocol/dist/parser.js:35:38)
at Socket.<anonymous> (/var/task/node_modules/pg-protocol/dist/index.js:11:42)
at Socket.emit (node:events:513:28)
at addChunk (node:internal/streams/readable:315:12)
at readableAddChunk (node:internal/streams/readable:289:9)
at Socket.Readable.push (node:internal/streams/readable:228:10)
at TCP.onStreamRead (node:internal/stream_base_commons:190:23) {
length: 161,
severity: 'FATAL',
code: '28000',
detail: undefined,
hint: undefined,
position: undefined,
internalPosition: undefined,
internalQuery: undefined,
where: undefined,
schema: undefined,
table: undefined,
column: undefined,
dataType: undefined,
constraint: undefined,
file: 'auth.c',
line: '542',
routine: 'ClientAuthentication'
}
END RequestId: 10b5a33e-6fa2-4ca9-a0b2-7e0c0eee1b2a
REPORT RequestId: 10b5a33e-6fa2-4ca9-a0b2-7e0c0eee1b2a Duration: 235.65 ms Billed Duration: 236 ms Memory Size: 128 MB Max Memory Used: 68 MB
XRAY TraceId: 1-66a1f463-56211eef534b12b66a2845f4 SegmentId: 55c5e28e6950da0a Sampled: true
Request ID
10b5a33e-6fa2-4ca9-a0b2-7e0c0eee1b2a
以下是在 SAM 模板中设置 lambda 的方式：
CreateUserLambda:
Type: AWS::Serverless::Function
Properties:
CodeUri: backend/database/
Handler: src/create-user/lambda_handler.handler
Runtime: nodejs16.x
Role: !GetAtt LambdaExecutionRole.Arn
VpcConfig:
SecurityGroupIds:
- !GetAtt DefaultSecurityGroup.GroupId
SubnetIds:
- !Ref DefaultSubnetA
- !Ref DefaultSubnetB
以下是数据库和安全组的设置方式的片段：
# AWS Secrets Manager for DB Credentials
DBInstanceSecret:
Type: AWS::SecretsManager::Secret
Properties:
Description: PostgreSQL RDS database credentials
GenerateSecretString:
SecretStringTemplate: '{"username": "postgres"}'
GenerateStringKey: "password"
PasswordLength: 16
ExcludeCharacters: '"@/\\'
Name: !Sub "${DbName}-credentials"
# VPC and Networking
DefaultVPC:
Type: AWS::EC2::VPC
Properties:
CidrBlock: "10.0.0.0/16"
EnableDnsSupport: true
EnableDnsHostnames: true
Tags:
- Key: Name
Value: "DefaultVPC"
InternetGateway:
Type: AWS::EC2::InternetGateway
Properties:
Tags:
- Key: Name
Value: "DefaultInternetGateway"
AttachGateway:
Type: AWS::EC2::VPCGatewayAttachment
Properties:
VpcId: !Ref DefaultVPC
InternetGatewayId: !Ref InternetGateway
PublicRouteTable:
Type: AWS::EC2::RouteTable
Properties:
VpcId: !Ref DefaultVPC
Tags:
- Key: Name
Value: "PublicRouteTable"
PublicRoute:
Type: AWS::EC2::Route
DependsOn: AttachGateway
Properties:
RouteTableId: !Ref PublicRouteTable
DestinationCidrBlock: "0.0.0.0/0"
GatewayId: !Ref InternetGateway
DefaultSubnetA:
Type: AWS::EC2::Subnet
Properties:
VpcId: !Ref DefaultVPC
CidrBlock: "10.0.1.0/24"
AvailabilityZone: !Select [0, !GetAZs ]
MapPublicIpOnLaunch: true
DefaultSubnetB:
Type: AWS::EC2::Subnet
Properties:
VpcId: !Ref DefaultVPC
CidrBlock: "10.0.2.0/24"
AvailabilityZone: !Select [1, !GetAZs ]
MapPublicIpOnLaunch: true
SubnetARouteTableAssociation:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
SubnetId: !Ref DefaultSubnetA
RouteTableId: !Ref PublicRouteTable
SubnetBRouteTableAssociation:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
SubnetId: !Ref DefaultSubnetB
RouteTableId: !Ref PublicRouteTable
DefaultDBSubnetGroup:
Type: AWS::RDS::DBSubnetGroup
Properties:
DBSubnetGroupDescription: "Subnet group for RDS using default subnets"
SubnetIds:
- !Ref DefaultSubnetA
- !Ref DefaultSubnetB
DefaultSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: "Default VPC security group"
VpcId: !Ref DefaultVPC
SecurityGroupIngress:
- IpProtocol: -1
CidrIp: "0.0.0.0/0"
SecurityGroupEgress:
- IpProtocol: -1
CidrIp: "0.0.0.0/0"
# RDS Database Instance
MyDbInstance:
Type: AWS::RDS::DBInstance
Properties:
DBInstanceClass: db.t3.micro
AllocatedStorage: 20
DBName: !Ref DbName
Engine: postgres
EngineVersion: 16.3
MasterUsername: "postgres"
MasterUserPassword: !Join ['', ['{{resolve:secretsmanager:', !Ref DBInstanceSecret, ':SecretString:password}}']]
VPCSecurityGroups:
- !GetAtt DefaultSecurityGroup.GroupId
DBSubnetGroupName: !Ref DefaultDBSubnetGroup
PubliclyAccessible: true
# API Gateway
ClientAPIGateway:
Type: AWS::ApiGateway::RestApi
Properties:
Name: ClientAPIGateway
Description: API Gateway for handling all client and Plaid API requests
# Cognito Authorizer for the API
CognitoAuthorizer:
Type: AWS::ApiGateway::Authorizer
Properties:
Name: CognitoAuthorizer
Type: COGNITO_USER_POOLS
IdentitySource: method.request.header.Authorization
RestApiId: !Ref ClientAPIGateway
ProviderARNs:
- !GetAtt CognitoUserPool.Arn
我已经尝试过的一些步骤：
默认情况下，RDS for PostgreSQL 使用并期望所有客户端使用 SSL/TLS 进行连接。https https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/PostgreSQL.Concepts.General.SSL.html
谈到最佳实践，不建议通过将参数从 1 更改为 0 来关闭 SSL。
首先，检查您当前的 AWS 区域并下载正确的 证书包 。您还可以从 证书颁发机构 .
其次，从代码中检查数据库配置，并尝试添加有关 SSL 的 ，例如
require: true
ca: <path-to-cert-bundle>
然后，通过在 AWS Lambda 函数（在 VPC 内）和 Amazon RDS for PostgreSQL 实例之间设置安全连接重试并测试您的代码。