网站建设联/百度关键词优化专家

简单思路演示代码

• 简单代码演示,基本思路（实际中需要像后面代码一样先洗牌，且每次shuffer后顺序不同）

a = np.arange(100)def batch_gen(data):  # 定义batch数据生成器idx = 0while True:if idx+10>100:idx=0start = idxidx += 10yield data[start:start+10]gen = batch_gen(a)for i in range(20):b = next(gen)  # 在循环中利用next()函数调用batch数据print(b)

• 代码运行结果

[0 1 2 3 4 5 6 7 8 9]
[10 11 12 13 14 15 16 17 18 19]
[20 21 22 23 24 25 26 27 28 29]
[30 31 32 33 34 35 36 37 38 39]
[40 41 42 43 44 45 46 47 48 49]
[50 51 52 53 54 55 56 57 58 59]
[60 61 62 63 64 65 66 67 68 69]
[70 71 72 73 74 75 76 77 78 79]
[80 81 82 83 84 85 86 87 88 89]
[90 91 92 93 94 95 96 97 98 99]
[0 1 2 3 4 5 6 7 8 9]
[10 11 12 13 14 15 16 17 18 19]
[20 21 22 23 24 25 26 27 28 29]
[30 31 32 33 34 35 36 37 38 39]
[40 41 42 43 44 45 46 47 48 49]
[50 51 52 53 54 55 56 57 58 59]
[60 61 62 63 64 65 66 67 68 69]
[70 71 72 73 74 75 76 77 78 79]
[80 81 82 83 84 85 86 87 88 89]
[90 91 92 93 94 95 96 97 98 99]

实际batch-generator代码

• 定义批量数据生成器

def shuffle_aligned_list(data):"""Shuffle arrays in a list by shuffling each array identically."""num = data[0].shape[0]p = np.random.permutation(num)return [d[p] for d in data]def batch_generator(data, batch_size, shuffle=True):"""Generate batches of data.Given a list of array-like objects, generate batches of a givensize by yielding a list of array-like objects corresponding to thesame slice of each input."""if shuffle:data = shuffle_aligned_list(data)batch_count = 0while True:if batch_count * batch_size + batch_size > len(data[0]):batch_count = 0if shuffle:data = shuffle_aligned_list(data)start = batch_count * batch_sizeend = start + batch_sizebatch_count += 1yield [d[start:end] for d in data]

• 实例化批量数据生成器

# make data
x = np.arange(1,51).reshape((10,5))
y = np.arange(1,11).reshape((-1,1))# define epochs and batch_size
batch_size = 5
epochs_num = (len(x)//batch_size)*epochs# define generator
batch_gen = batch_generator([x,y],batch_size) # assign batch_gen as batch generator

• 利用next()循环语句中调用gen

for i in range(20):batch_x,batch_y = next(batch_gen)if int(i%2)==0:print('Epoch %d'%int(i/2))print('The %d Batch_y:\n '%(i+1),batch_y.reshape((1,-1)),'\n')

Epoch 1
The 1 Batch_y:[[ 5  2  7 10  4]] The 2 Batch_y:[[1 3 6 9 8]] Epoch 2
The 3 Batch_y:[[2 3 6 8 1]] The 4 Batch_y:[[ 5  9 10  4  7]] Epoch 3
The 5 Batch_y:[[ 2 10  1  3  8]] The 6 Batch_y:[[7 9 6 5 4]] Epoch 4
The 7 Batch_y:[[2 4 5 3 6]] The 8 Batch_y:[[ 1  7  9  8 10]] Epoch 5
The 9 Batch_y:[[10  4  7  5  2]] The 10 Batch_y:[[1 8 9 6 3]] Epoch 6
The 11 Batch_y:[[6 3 8 9 7]] The 12 Batch_y:[[ 1 10  2  4  5]] Epoch 7
The 13 Batch_y:[[1 3 4 5 8]] The 14 Batch_y:[[ 9  6  7 10  2]] Epoch 8
The 15 Batch_y:[[ 7 10  5  2  9]] The 16 Batch_y:[[6 4 3 1 8]] Epoch 9
The 17 Batch_y:[[ 3 10  2  4  8]] The 18 Batch_y:[[5 6 1 7 9]] Epoch 10
The 19 Batch_y:[[ 1  4  7 10  3]] The 20 Batch_y:[[8 9 6 5 2]]