基于electron开发在线课堂

此文不讲整体项目思路,只是对该项目开发的一些踩坑总结,备忘用。

相关链接

音视频流

  • webrtc samples github
  • webrtc samples
  • 如何绘制麦克风实时音量图谱

腾讯云

  • 云直播 LIVE 控制台
  • 即时通信 IM 控制台
  • 实时音视频 TRTC 控制台
  • web 设备检测
  • IM Web SDK
  • TRTC Web SDK
  • TRTC ELECTRON SDK
  • 在线教育互动课堂 SAAS
  • trtc-electron-education API

注:

  1. 在线课堂 electron 应用可以基于实时互动课堂(Electron)开发,本质是实时音视频trtc+即时通讯im的组合。
  2. 虽然 electron 本质是打包 web 应用,但由于腾讯云 SDK 实现问题,TRTC Web SDK 并不能用在 electron 应用里,可能会有兼容问题。

electron

  • electron
  • electron 简单介绍
  • electron BrowserWindow API
  • electron react 模板
  • 打包器 electron-builder
  • 自动更新 electron-updater

记录点

异步获取系统设备

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
export const getDeviceList = async (type: MediaType): Promise<[Device | null, Device[]]> => {
if (!deviceInfo) {
const devices = await navigator.mediaDevices.enumerateDevices();
const deviceMap: DeviceMap = {
audiooutput: 'speaker',
audioinput: 'microphone',
videoinput: 'camera'
};
const curDeviceInfo: DeviceInfo = {};

devices.forEach(({ deviceId, kind, label }) => {
if (!curDeviceInfo[deviceMap[kind]]) {
curDeviceInfo[deviceMap[kind]] = [];
}
curDeviceInfo[deviceMap[kind]]!.push({
deviceId,
kind,
label,
isCurrent: deviceId === 'default'
});
});
Object.values(curDeviceInfo).forEach((item: Device[]) => {
if (item.length === 1) {
item[0].isCurrent = true;
}
});
deviceInfo = curDeviceInfo;
}
const list = deviceInfo[type]!;
if (!list || list.length === 0) {
return [null, []];
}
const curDevice = list!.filter((device) => device.isCurrent)[0];
return [curDevice, list];
};

选择对应扬声器

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
const selectSpeackerDevice = (current: HTMLMediaElement): ((id: string) => void) => (id: string) => {
const currentId = counterId;
current
.setSinkId(id)
.then(() => {
current.currentTime = 0;
if (currentId !== counterId) {
current.pause();
return;
}
current.play();
})
.catch((e) => {
message.error(`获取扬声器失败!`);
console.log(e);
});
};

清除 media 流及动画

1
2
3
4
5
6
7
export const clearMediaAndAnimate = () => {
if (mediaStream) {
mediaStream.getTracks()[0].stop();
}
cancelAnimationFrame(animate);
counterId += 1;
};

选择对应麦克风并绘制音量图谱

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
const setCanvas = (current: HTMLCanvasElement) => {
const { width, height } = current;
const canvasCtx = current.getContext('2d')!;
canvasCtx.clearRect(0, 0, width, height);
canvasCtx.fillStyle = '#fff';
canvasCtx.fillRect(0, 0, width, height);

return (info: number) => {
canvasCtx.clearRect(0, 0, width, height);
canvasCtx.fillRect(0, 0, info, height);

for (let i = 0; i < info; i += 1) {
canvasCtx.beginPath();
canvasCtx.lineWidth = 2;
canvasCtx.strokeStyle = i % 2 ? `#437BFF` : '#fff';
canvasCtx.moveTo(i, 0);
canvasCtx.lineTo(i, height);
canvasCtx.stroke();
}
};
};

const selectMicrophoneDevice = (current: HTMLCanvasElement): ((id: string) => void) => (id: string) => {
const currentId = counterId;
const constraints = {
audio: { deviceId: { exact: id } }
};
const drawCanvas = setCanvas(current);
const audioCtx = new AudioContext();
const analyser = audioCtx.createAnalyser(); // 频率及时间域分析器
analyser.fftSize = 256;
let source;

navigator.mediaDevices
.getUserMedia(constraints)
.then((stream) => {
mediaStream = stream;
if (currentId !== counterId) {
clearMediaAndAnimate();
return;
}
source = audioCtx.createMediaStreamSource(stream); // 创建源
source.connect(analyser);
const dataArray = new Uint8Array(analyser.frequencyBinCount);
const draw = () => {
animate = requestAnimationFrame(draw);
analyser.getByteFrequencyData(dataArray);
drawCanvas(dataArray[0]);
};
draw();
})
.catch((e) => {
message.error(`获取麦克风失败!`);
console.log(e);
clearMediaAndAnimate();
});
};

选择对应摄像头

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
const selectCameraDevice = (current: HTMLVideoElement): ((id: string) => void) => (id: string) => {
const currentId = counterId;
const constraints = {
video: { deviceId: { exact: id } },
};
navigator.mediaDevices
.getUserMedia(constraints)
.then((stream) => {
mediaStream = stream;
if (currentId !== counterId) {
clearMediaAndAnimate();
return;
}
current.srcObject = stream;
})
.catch((e) => {
message.error(`获取视频流失败!`);
console.log(e);
clearMediaAndAnimate();
});
};

其中通过 counterId 保留当前最新流,异步丢弃之前旧流。

electron-builder.json

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
{
"productName": "electron客户端",
"appId": "cn.electron.korey",
"copyright": "Copyright © 2021 korey",
"asar": true,
"compression": "maximum", //若用 store,则打包速度加快,但打包体积变大
"nsis": {
"oneClick": false, //取消一键安装
"allowElevation": true,
"allowToChangeInstallationDirectory": true,
"installerIcon": "./resources/icons/icon.ico", // 256*256
"uninstallerIcon": "./resources/icons/icon.ico",
"installerHeaderIcon": "./resources/icons/icon.ico",
"createDesktopShortcut": true,
"createStartMenuShortcut": true,
"shortcutName": "electron客户端"
},
"files": ["dist/", "node_modules/", "app.html", "main.prod.js", "main.prod.js.map", "package.json"],
"dmg": {
"contents": [
{
"x": 130,
"y": 220
},
{
"x": 410,
"y": 220,
"type": "link",
"path": "/Applications"
}
]
},
"win": {
"extraFiles": [
{
"from": "node_modules/trtc-electron-sdk/build/Release", //将 .node 文件复制过去,没有这个程序将报错
"to": "."
}
],
"target": {
"target": "nsis",
"arch": "x64"
},
"icon": "./resources/icons/icon.ico" //256*256 ico格式,未配 icon 则 win 打包报错
},
"mac": {
"category": "zhibojiaoyu.app",
"extraFiles": [
{
"from": "node_modules/trtc-electron-sdk/build/Release", //同上
"to": "./Frameworks"
}
]
},
"linux": {
"target": ["deb", "rpm", "AppImage"],
"category": "Development"
},
"directories": {
"buildResources": "resources",
"output": "release"
},
"publish": {
"provider": "generic",
"channel": "latest",
"url": "http://cdn.flqin.com/electron客户端-1.0.0.dmg",
"private": false
},
"electronDownload": {
"mirror": "https://npm.taobao.org/mirrors/electron/"
}
}

腾讯云相关

  • 直播群 AVChatRoom(需求大于 6000 人)不支持历史消息存储。群组系统对比
  • startScreenCapture 开启屏幕推流后,可通过 setSubStreamMixVolume 设置麦克风和屏幕里音源大小比例。win 上需异步调用 startSystemAudioLoopback 才能采集到屏幕里音源。其中默认摄像头为主流,屏幕为辅流。
  • IM 群组已存在,除直播群需要同时调 joinGroup 以外,其他类型再次 createGroup 会直接进入该群组。
  • trtc enterRoom roomId 取值范围 1~4294967295
  • trtc getScreenCaptureSourcesmac os big sur 版本返回的 screenList.thumbBGRA 里的 width*height*4 不等于 buffer.length 导致程序报错,等待腾讯云修复。
  • 窗口置顶:setAlwaysOnTop(true, 'pop-up-menu'), 一定要有 pop-up-menu 参数,因为在 win 上无此参数时分享全屏屏幕时,拖动置顶窗口会意外置底。

sdk node 支持

webpack 需配置解析腾讯云 sdk .noderules

1
2
3
4
5
6
7
8
{
"test": /\.node$/,
"loader": "native-ext-loader",
"options": {
"emit": false,
"rewritePath": process.env.NODE_ENV === "production" ? "./" : "node_modules/trtc-electron-sdk/build/Release/"
}
}

BrowserWindow 配置里需加上 webPreferences: {nodeIntegration: true}

其他

  • 使用 setBounds 代替 setSize,因为 setSizewin 上多次调用会失效。
  • CSS 中指定 -webkit-app-region: drag 来告诉 Electron 哪些区域可拖拽。