update to manifest-v3
This commit is contained in:
makaveli10
@@ -1,21 +0,0 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2017 Justice Yen
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
@@ -2,6 +2,8 @@
|
||||
|
||||
Audio Transcription is a Chrome extension that allows users to capture any audio playing on the current tab and transcribe it using OpenAI-whisper in real time. Users will have the option to do voice activity detection as well to not send audio to server when there is no speech.
|
||||
|
||||
We use OpenAI-whisper model to process the audio continuously and send the transcription back to the client. We apply a few optimizations on top of OpenAI's implementation to improve performance and run it faster in a real-time manner. To this end, we used [faster-whisper](https://github.com/guillaumekln/faster-whisper) which is 4x faster than OpenAI's implementation.
|
||||
|
||||
## Loading the Extension
|
||||
- Open the Google Chrome browser.
|
||||
- Type chrome://extensions in the address bar and press Enter.
|
||||
@@ -16,43 +18,18 @@ Audio Transcription is a Chrome extension that allows users to capture any audio
|
||||
This Chrome extension allows you to send audio from your browser to a server for transcribing the audio in real time. It can also incorporate voice activity detection on the client side to detect when speech is present, and it continuously receives transcriptions of the spoken content from the server. You can select from the options menu if you want to run the speech recognition.
|
||||
|
||||
|
||||
## Running the Whisper-live server
|
||||
For a detailed overview of how to run the server to leverage real time transcriptions with OpenAI-whisper, use [whisper-live](https://github.com/collabora/whisper-live) to setup your own server.
|
||||
|
||||
|
||||
## Options
|
||||
Several options are able to be changed in the extension:
|
||||
- 'Mute tabs that are being captured' allows the extension to force any tabs currently being captured to be muted on the system's audio output, but still have its audio captured and encoded to the resulting file.
|
||||
- 'Maximum capture time' changes the amount of time the extension will capture audio for before timing out, and has a limit to prevent exceeding Chrome's memory limit.
|
||||
- 'Output file format' allows users to choose whether the resulting file will be encoded into .wav or .mp3
|
||||
- 'MP3 Quality' is only applicable for .mp3 encodings, and will change the bitrate of the encode. (Low: 96 kbps, Medium: 192 kbps, High: 320 kbps)
|
||||
- 'Enable Voice Activity detection' allows users to run a voice activity detection model before sending audio to a server hosting the OpenAI-whisper model for transcriptions.
|
||||
|
||||
|
||||
## Implementation Details
|
||||
|
||||
### Capturing Audio
|
||||
To capture the audio in the current tab, I used the chrome `tabCapture` API to obtain a `MediaStream` object of the current tab. Next I used the `MediaStream` object to initialize a recorder that will encode the stream into a .wav file using the `Recorder.js` library.
|
||||
To capture the audio in the current tab, we used the chrome `tabCapture` API to obtain a `MediaStream` object of the current tab.
|
||||
|
||||
### Audio Transcription
|
||||
We use OpenAI-whisper model to process the audio continuously and send the transcription back to the client. We apply a few optimizations on top of OpenAI's implementation to improve performance and run it faster in a real-time manner. To this end, we used [faster-whisper](https://github.com/guillaumekln/faster-whisper) which is 4x faster than OpenAI's implementation.
|
||||
|
||||
### Voice Activity Detection
|
||||
For VAD, we use [silero-vad](https://github.com/snakers4/silero-vad) which is both efficient and accurate for detecting voice activity. It takes around ```1ms``` to process a single audio chunk of ```30ms```. We use the ONNX model in the browser to only send the audio to the server when there is a voice activity.
|
||||
|
||||
### Tab Management
|
||||
To allow audio capture on multiple tabs simultaneously, I stored the `tabId` of each tab being captured into the `sessionStorage` object. When a `stopCapture` command is issued, the extension will check whether the current tab is the same as the tab that the capture was started on, and only stop the specific instance of the capture on the current tab.
|
||||
|
||||
|
||||
### Audio Playback During Capture
|
||||
By default, using `tabCapture` will mute the audio on the current tab in order for the capture to take place. To allow audio to continue playing during the capture, I created an `Audio` object which has its source linked to the ongoing stream that is being captured. In the options menu, users will have the option to keep the tab muted or unmuted during the capture.
|
||||
### Getting Started
|
||||
- Make sure the transcription server is running properly. To know more about how to start the server, see the [documentation here](https://github.com/collabora/whisper-live).
|
||||
- Just click on the Chrome Extension which should show 2 options
|
||||
- **Start Capture** : Starts capturing the audio in the current tab and sends the captured audio to the server for transcription. This also creates an element to show the transcriptions recieved from the server on the current tab.
|
||||
- **Stop Capture** - Stops capturing the audio.
|
||||
|
||||
|
||||
## Limitations
|
||||
This extension requires an internet connection to stream audio and receive transcriptions. The accuracy of the transcriptions may vary depending on the audio quality and the performance of the server-side transcription service. The extension may consume additional system resources while running, especially when streaming audio.
|
||||
|
||||
## License
|
||||
This extension is provided as-is, without any warranty or guarantee of its performance or suitability for any particular purpose. The developers of this extension shall not be held responsible for any damages or losses incurred while using this extension.
|
||||
This extension uses LAME MP3 encoder, licensed LGPL.
|
||||
Everything else is under the MIT License.
|
||||
|
||||
|
||||
@@ -1,3 +0,0 @@
|
||||
<script src="https://cdn.jsdelivr.net/npm/onnxruntime-web/dist/ort.min.js"></script>
|
||||
<script src="background.js"></script>
|
||||
<script src="worker.js"></script>
|
||||
+173
-351
@@ -1,368 +1,190 @@
|
||||
const extend = function() { //helper function to merge objects
|
||||
let target = arguments[0],
|
||||
sources = [].slice.call(arguments, 1);
|
||||
for (let i = 0; i < sources.length; ++i) {
|
||||
let src = sources[i];
|
||||
for (key in src) {
|
||||
let val = src[key];
|
||||
target[key] = typeof val === "object"
|
||||
? extend(typeof target[key] === "object" ? target[key] : {}, val)
|
||||
: val;
|
||||
}
|
||||
}
|
||||
return target;
|
||||
};
|
||||
/**
|
||||
* Removes a tab with the specified tab ID in Google Chrome.
|
||||
* @param {number} tabId - The ID of the tab to be removed.
|
||||
* @returns {Promise<void>} A promise that resolves when the tab is successfully removed or fails to remove.
|
||||
*/
|
||||
function removeChromeTab(tabId) {
|
||||
return new Promise((resolve) => {
|
||||
chrome.tabs.remove(tabId)
|
||||
.then(resolve)
|
||||
.catch(resolve);
|
||||
});
|
||||
}
|
||||
|
||||
const WORKER_FILE = {
|
||||
wav: "WavWorker.js",
|
||||
mp3: "Mp3Worker.js"
|
||||
};
|
||||
|
||||
// default configs
|
||||
const CONFIGS = {
|
||||
workerDir: "/workers/", // worker scripts dir (end with /)
|
||||
numChannels: 2, // number of channels
|
||||
encoding: "wav", // encoding (can be changed at runtime)
|
||||
|
||||
// runtime options
|
||||
options: {
|
||||
timeLimit: 1200, // recording time limit (sec)
|
||||
encodeAfterRecord: true, // process encoding after recording
|
||||
progressInterval: 1000, // encoding progress report interval (millisec)
|
||||
bufferSize: 4096, // buffer size (use browser default)
|
||||
|
||||
// encoding-specific options
|
||||
wav: {
|
||||
mimeType: "audio/wav"
|
||||
},
|
||||
mp3: {
|
||||
mimeType: "audio/mpeg",
|
||||
bitRate: 192 // (CBR only): bit rate = [64 .. 320]
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
class Recorder {
|
||||
|
||||
constructor(source, configs) { //creates audio context from the source and connects it to the worker
|
||||
extend(this, CONFIGS, configs || {});
|
||||
this.speech_threshold = 0.4
|
||||
this.context = source.context;
|
||||
if (this.context.createScriptProcessor == null)
|
||||
this.context.createScriptProcessor = this.context.createJavaScriptNode;
|
||||
this.input = this.context.createGain();
|
||||
source.connect(this.input);
|
||||
this.buffer = [];
|
||||
this.initWorker();
|
||||
}
|
||||
|
||||
isRecording() {
|
||||
return this.processor != null;
|
||||
}
|
||||
|
||||
setEncoding(encoding) {
|
||||
if(!this.isRecording() && this.encoding !== encoding) {
|
||||
this.encoding = encoding;
|
||||
this.initWorker();
|
||||
}
|
||||
}
|
||||
|
||||
setOptions(options) {
|
||||
if (!this.isRecording()) {
|
||||
extend(this.options, options);
|
||||
this.worker.postMessage({ command: "options", options: this.options});
|
||||
}
|
||||
}
|
||||
|
||||
async startRecording(doVad) {
|
||||
if(!this.isRecording()) {
|
||||
let numChannels = this.numChannels;
|
||||
let buffer = this.buffer;
|
||||
let worker = this.worker;
|
||||
|
||||
// initialize onnx model
|
||||
const session = await ort.InferenceSession.create('./silero_vad.onnx');
|
||||
var h = new Array(128);
|
||||
for (let i = 0; i < h.length; i++) {
|
||||
h[i] = 0;
|
||||
/**
|
||||
* Executes a script file in a specific tab in Google Chrome.
|
||||
* @param {number} tabId - The ID of the tab where the script should be executed.
|
||||
* @param {string} file - The file path or URL of the script to be executed.
|
||||
* @returns {Promise<void>} A promise that resolves when the script is successfully executed or fails to execute.
|
||||
*/
|
||||
function executeScriptInTab(tabId, file) {
|
||||
return new Promise((resolve) => {
|
||||
chrome.scripting.executeScript(
|
||||
{
|
||||
target: { tabId },
|
||||
files: [file],
|
||||
}, () => {
|
||||
resolve();
|
||||
}
|
||||
var c = new Array(128);
|
||||
for (let i = 0; i < h.length; i++) {
|
||||
c[i] = 0;
|
||||
}
|
||||
|
||||
const sr = new BigInt64Array(1)
|
||||
sr[0] = BigInt(16000);
|
||||
const srate = new ort.Tensor('int64', sr, [1]);
|
||||
let speech_prob = undefined;
|
||||
const vad_infer = async (feed_dict) => {
|
||||
// feed inputs and run
|
||||
try{
|
||||
const results = await session.run(feed_dict);
|
||||
);
|
||||
});
|
||||
}
|
||||
|
||||
// update states
|
||||
h = results.hn.data
|
||||
c = results.cn.data
|
||||
speech_prob = results.output.data
|
||||
} catch(e) {
|
||||
console.log(e)
|
||||
}
|
||||
}
|
||||
this.processor = this.context.createScriptProcessor(
|
||||
this.options.bufferSize,
|
||||
this.numChannels, this.numChannels);
|
||||
this.input.connect(this.processor);
|
||||
this.processor.connect(this.context.destination);
|
||||
this.processor.onaudioprocess = function(event) {
|
||||
for (var ch = 0; ch < numChannels; ++ch)
|
||||
buffer[ch] = event.inputBuffer.getChannelData(ch);
|
||||
|
||||
// voice activity detection inference
|
||||
const audioBuffer = new ort.Tensor('float32', buffer[0], [1, buffer[0].length]);
|
||||
const hh = new ort.Tensor('float32', h, [2, 1, 64]);
|
||||
const hc = new ort.Tensor('float32', c, [2, 1, 64]);
|
||||
const feeds = { input: audioBuffer, sr: srate, h: hh, c: hc};
|
||||
|
||||
// feed inputs and run
|
||||
if (doVad) {
|
||||
vad_infer(feeds)
|
||||
if (speech_prob > 0.4) {
|
||||
worker.postMessage({ command: "record", buffer: buffer });
|
||||
}
|
||||
else
|
||||
console.log("no speech found: " + speech_prob)
|
||||
}
|
||||
else
|
||||
worker.postMessage({ command: "record", buffer: buffer });
|
||||
};
|
||||
this.worker.postMessage({
|
||||
command: "start",
|
||||
bufferSize: this.processor.bufferSize
|
||||
});
|
||||
this.startTime = Date.now();
|
||||
}
|
||||
}
|
||||
|
||||
cancelRecording() {
|
||||
if(this.isRecording()) {
|
||||
this.input.disconnect();
|
||||
this.processor.disconnect();
|
||||
delete this.processor;
|
||||
this.worker.postMessage({ command: "cancel" });
|
||||
}
|
||||
}
|
||||
|
||||
finishRecording() {
|
||||
if (this.isRecording()) {
|
||||
this.input.disconnect();
|
||||
this.processor.disconnect();
|
||||
delete this.processor;
|
||||
this.worker.postMessage({ command: "finish" });
|
||||
}
|
||||
}
|
||||
|
||||
cancelEncoding() {
|
||||
if (this.options.encodeAfterRecord)
|
||||
if (!this.isRecording()) {
|
||||
this.onEncodingCanceled(this);
|
||||
this.initWorker();
|
||||
}
|
||||
}
|
||||
|
||||
initWorker() {
|
||||
if (this.worker != null)
|
||||
this.worker.terminate();
|
||||
this.onEncoderLoading(this, this.encoding);
|
||||
this.worker = new Worker(this.workerDir + WORKER_FILE[this.encoding]);
|
||||
let _this = this;
|
||||
this.worker.onmessage = function(event) {
|
||||
let data = event.data;
|
||||
switch (data.command) {
|
||||
case "transcription":
|
||||
chrome.tabs.getSelected(null, function(tab) {
|
||||
chrome.tabs.sendMessage(tab.id, { message: data.text });
|
||||
});
|
||||
break;
|
||||
case "loaded":
|
||||
_this.onEncoderLoaded(_this, _this.encoding);
|
||||
break;
|
||||
case "timeout":
|
||||
_this.onTimeout(_this);
|
||||
break;
|
||||
case "progress":
|
||||
_this.onEncodingProgress(_this, data.progress);
|
||||
break;
|
||||
case "complete":
|
||||
_this.onComplete(_this, data.blob);
|
||||
}
|
||||
}
|
||||
this.worker.postMessage({
|
||||
command: "init",
|
||||
config: {
|
||||
sampleRate: this.context.sampleRate,
|
||||
numChannels: this.numChannels
|
||||
/**
|
||||
* Opens the options page of the Chrome extension in a new pinned tab.
|
||||
* @returns {Promise<chrome.tabs.Tab>} A promise that resolves with the created tab object.
|
||||
*/
|
||||
function openExtensionOptions() {
|
||||
return new Promise((resolve) => {
|
||||
chrome.tabs.create(
|
||||
{
|
||||
pinned: true,
|
||||
active: false,
|
||||
url: `chrome-extension://${chrome.runtime.id}/options.html`,
|
||||
},
|
||||
options: this.options
|
||||
(tab) => {
|
||||
resolve(tab);
|
||||
}
|
||||
);
|
||||
});
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* Retrieves the value associated with the specified key from the local storage in Google Chrome.
|
||||
* @param {string} key - The key of the value to retrieve from the local storage.
|
||||
* @returns {Promise<any>} A promise that resolves with the retrieved value from the local storage.
|
||||
*/
|
||||
function getLocalStorageValue(key) {
|
||||
return new Promise((resolve) => {
|
||||
chrome.storage.local.get([key], (result) => {
|
||||
resolve(result[key]);
|
||||
});
|
||||
}
|
||||
|
||||
onEncoderLoading(recorder, encoding) {}
|
||||
onEncoderLoaded(recorder, encoding) {}
|
||||
onTimeout(recorder) {}
|
||||
onEncodingProgress(recorder, progress) {}
|
||||
onEncodingCanceled(recorder) {}
|
||||
onComplete(recorder, blob) {}
|
||||
|
||||
});
|
||||
}
|
||||
|
||||
const audioCapture = (timeLimit, muteTab, format, quality, limitRemoved, doVad) => {
|
||||
chrome.tabCapture.capture({audio: true}, (stream) => { // sets up stream for capture
|
||||
let startTabId; //tab when the capture is started
|
||||
let timeout;
|
||||
let completeTabID; //tab when the capture is stopped
|
||||
let audioURL = null; //resulting object when encoding is completed
|
||||
chrome.tabs.query({active:true, currentWindow: true}, (tabs) => startTabId = tabs[0].id) //saves start tab
|
||||
const liveStream = stream;
|
||||
const audioCtx = new AudioContext({sampleRate: 16000});
|
||||
const source = audioCtx.createMediaStreamSource(stream);
|
||||
let mediaRecorder = new Recorder(source); //initiates the recorder based on the current stream
|
||||
mediaRecorder.setEncoding(format); //sets encoding based on options
|
||||
if(limitRemoved) { //removes time limit
|
||||
mediaRecorder.setOptions({timeLimit: 10800});
|
||||
|
||||
/**
|
||||
* Sends a message to a specific tab in Google Chrome.
|
||||
* @param {number} tabId - The ID of the tab to send the message to.
|
||||
* @param {any} data - The data to be sent as the message.
|
||||
* @returns {Promise<any>} A promise that resolves with the response from the tab.
|
||||
*/
|
||||
function sendMessageToTab(tabId, data) {
|
||||
return new Promise((resolve) => {
|
||||
chrome.tabs.sendMessage(tabId, data, (response) => {
|
||||
resolve(response);
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* Delays the execution for a specified duration.
|
||||
* @param {number} ms - The duration to sleep in milliseconds (default: 0).
|
||||
* @returns {Promise<void>} A promise that resolves after the specified duration.
|
||||
*/
|
||||
function delayExecution(ms = 0) {
|
||||
return new Promise((resolve) => setTimeout(resolve, ms));
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* Sets a value associated with the specified key in the local storage of Google Chrome.
|
||||
* @param {string} key - The key to set in the local storage.
|
||||
* @param {any} value - The value to associate with the key in the local storage.
|
||||
* @returns {Promise<any>} A promise that resolves with the value that was set in the local storage.
|
||||
*/
|
||||
function setLocalStorageValue(key, value) {
|
||||
return new Promise((resolve) => {
|
||||
chrome.storage.local.set(
|
||||
{
|
||||
[key]: value,
|
||||
}, () => {
|
||||
resolve(value);
|
||||
}
|
||||
);
|
||||
});
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* Retrieves the tab object with the specified tabId.
|
||||
* @param {number} tabId - The ID of the tab to retrieve.
|
||||
* @returns {Promise<object>} - A Promise that resolves to the tab object.
|
||||
*/
|
||||
async function getTab(tabId) {
|
||||
return new Promise((resolve) => {
|
||||
chrome.tabs.get(tabId, (tab) => {
|
||||
resolve(tab);
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* Starts the capture process for the specified tab.
|
||||
* @param {number} tabId - The ID of the tab to start capturing.
|
||||
* @returns {Promise<void>} - A Promise that resolves when the capture process is started successfully.
|
||||
*/
|
||||
async function startCapture(tabId) {
|
||||
const optionTabId = await getLocalStorageValue("optionTabId");
|
||||
if (optionTabId) {
|
||||
await removeChromeTab(optionTabId);
|
||||
}
|
||||
|
||||
try {
|
||||
const currentTab = await getTab(tabId);
|
||||
if (currentTab.audible) {
|
||||
await setLocalStorageValue("currentTabId", currentTab.id);
|
||||
await executeScriptInTab(currentTab.id, "content.js");
|
||||
await delayExecution(500);
|
||||
|
||||
const optionTab = await openExtensionOptions();
|
||||
|
||||
await setLocalStorageValue("optionTabId", optionTab.id);
|
||||
await delayExecution(500);
|
||||
|
||||
await sendMessageToTab(optionTab.id, {
|
||||
type: "start_capture",
|
||||
data: { currentTabId: currentTab.id },
|
||||
});
|
||||
} else {
|
||||
mediaRecorder.setOptions({timeLimit: timeLimit/1000});
|
||||
console.log("No Audio");
|
||||
}
|
||||
if(format === "mp3") {
|
||||
mediaRecorder.setOptions({mp3: {bitRate: quality}});
|
||||
}
|
||||
mediaRecorder.startRecording(doVad);
|
||||
|
||||
function onStopCommand(command) { //keypress
|
||||
if (command === "stop") {
|
||||
stopCapture();
|
||||
}
|
||||
}
|
||||
function onStopClick(request) { //click on popup
|
||||
if(request === "stopCapture") {
|
||||
stopCapture();
|
||||
} else if (request === "cancelCapture") {
|
||||
cancelCapture();
|
||||
} else if (request.cancelEncodeID) {
|
||||
if(request.cancelEncodeID === startTabId && mediaRecorder) {
|
||||
mediaRecorder.cancelEncoding();
|
||||
}
|
||||
}
|
||||
}
|
||||
chrome.commands.onCommand.addListener(onStopCommand);
|
||||
chrome.runtime.onMessage.addListener(onStopClick);
|
||||
mediaRecorder.onComplete = (recorder, blob) => {
|
||||
audioURL = window.URL.createObjectURL(blob);
|
||||
if(completeTabID) {
|
||||
chrome.tabs.sendMessage(completeTabID, {type: "encodingComplete", audioURL});
|
||||
}
|
||||
mediaRecorder = null;
|
||||
}
|
||||
mediaRecorder.onEncodingProgress = (recorder, progress) => {
|
||||
if(completeTabID) {
|
||||
chrome.tabs.sendMessage(completeTabID, {type: "encodingProgress", progress: progress});
|
||||
}
|
||||
}
|
||||
|
||||
const stopCapture = function() {
|
||||
let endTabId;
|
||||
//check to make sure the current tab is the tab being captured
|
||||
chrome.tabs.query({active: true, currentWindow: true}, (tabs) => {
|
||||
endTabId = tabs[0].id;
|
||||
if(mediaRecorder && startTabId === endTabId){
|
||||
mediaRecorder.finishRecording();
|
||||
chrome.tabs.create({url: "complete.html"}, (tab) => {
|
||||
completeTabID = tab.id;
|
||||
let completeCallback = () => {
|
||||
chrome.tabs.sendMessage(tab.id, {type: "createTab", format: format, audioURL, startID: startTabId});
|
||||
}
|
||||
setTimeout(completeCallback, 500);
|
||||
});
|
||||
closeStream(endTabId);
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
const cancelCapture = function() {
|
||||
let endTabId;
|
||||
chrome.tabs.query({active: true, currentWindow: true}, (tabs) => {
|
||||
endTabId = tabs[0].id;
|
||||
if(mediaRecorder && startTabId === endTabId){
|
||||
mediaRecorder.cancelRecording();
|
||||
closeStream(endTabId);
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
//removes the audio context and closes recorder to save memory
|
||||
const closeStream = function(endTabId) {
|
||||
chrome.commands.onCommand.removeListener(onStopCommand);
|
||||
chrome.runtime.onMessage.removeListener(onStopClick);
|
||||
mediaRecorder.onTimeout = () => {};
|
||||
audioCtx.close();
|
||||
liveStream.getAudioTracks()[0].stop();
|
||||
sessionStorage.removeItem(endTabId);
|
||||
chrome.runtime.sendMessage({captureStopped: endTabId});
|
||||
}
|
||||
|
||||
mediaRecorder.onTimeout = stopCapture;
|
||||
|
||||
if(!muteTab) {
|
||||
let audio = new Audio();
|
||||
audio.srcObject = liveStream;
|
||||
audio.play();
|
||||
}
|
||||
});
|
||||
} catch (error) {
|
||||
console.error("Error occurred while starting capture:", error);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* Stops the capture process and performs cleanup.
|
||||
* @returns {Promise<void>} - A Promise that resolves when the capture process is stopped successfully.
|
||||
*/
|
||||
async function stopCapture() {
|
||||
const optionTabId = await getLocalStorageValue("optionTabId");
|
||||
const currentTabId = await getLocalStorageValue("currentTabId");
|
||||
|
||||
//sends reponses to and from the popup menu
|
||||
chrome.runtime.onMessage.addListener((request, sender, sendResponse) => {
|
||||
if (request.currentTab && sessionStorage.getItem(request.currentTab)) {
|
||||
sendResponse(sessionStorage.getItem(request.currentTab));
|
||||
} else if (request.currentTab){
|
||||
sendResponse(false);
|
||||
} else if (request === "startCapture") {
|
||||
startCapture();
|
||||
}
|
||||
});
|
||||
|
||||
const startCapture = function() {
|
||||
chrome.tabs.query({active: true, currentWindow: true}, (tabs) => {
|
||||
// CODE TO BLOCK CAPTURE ON YOUTUBE, DO NOT REMOVE
|
||||
// if(tabs[0].url.toLowerCase().includes("youtube")) {
|
||||
// chrome.tabs.create({url: "error.html"});
|
||||
// } else {
|
||||
if(!sessionStorage.getItem(tabs[0].id)) {
|
||||
sessionStorage.setItem(tabs[0].id, Date.now());
|
||||
chrome.storage.sync.get({
|
||||
maxTime: 1200000,
|
||||
muteTab: false,
|
||||
format: "mp3",
|
||||
quality: 192,
|
||||
limitRemoved: false,
|
||||
doVad: false
|
||||
}, (options) => {
|
||||
let time = options.maxTime;
|
||||
if(time > 1200000) {
|
||||
time = 1200000
|
||||
}
|
||||
audioCapture(time, options.muteTab, options.format, options.quality, options.limitRemoved, options.doVad);
|
||||
});
|
||||
chrome.runtime.sendMessage({captureStarted: tabs[0].id, startTime: Date.now()});
|
||||
}
|
||||
// }
|
||||
});
|
||||
};
|
||||
|
||||
|
||||
chrome.commands.onCommand.addListener((command) => {
|
||||
if (command === "start") {
|
||||
startCapture();
|
||||
if (optionTabId) {
|
||||
res = await sendMessageToTab(currentTabId, {
|
||||
type: "STOP"
|
||||
});
|
||||
await removeChromeTab(optionTabId);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* Listens for messages from the runtime and performs corresponding actions.
|
||||
* @param {Object} message - The message received from the runtime.
|
||||
*/
|
||||
chrome.runtime.onMessage.addListener((message) => {
|
||||
if (message.action === "startCapture") {
|
||||
startCapture(message.tabId);
|
||||
} else if (message.action === "stopCapture") {
|
||||
stopCapture();
|
||||
}
|
||||
});
|
||||
|
||||
@@ -1,71 +0,0 @@
|
||||
.header {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
padding-bottom: 15px;
|
||||
padding-left: 20px;
|
||||
border-bottom: 2px solid darkred;
|
||||
}
|
||||
|
||||
.header-title {
|
||||
padding: 0 5px;
|
||||
}
|
||||
|
||||
.progress {
|
||||
margin: 50px 20px 10px 20px;
|
||||
font-size: 16px;
|
||||
display: flex;
|
||||
}
|
||||
|
||||
.notes {
|
||||
margin: 0 20px;
|
||||
}
|
||||
|
||||
#progressContainer {
|
||||
margin-left: 5px;
|
||||
width: 500px;
|
||||
height: 18px;
|
||||
background-color: lavenderblush;
|
||||
}
|
||||
|
||||
#encodeProgress {
|
||||
height: 100%;
|
||||
width: 0%;
|
||||
background-color: darkred;
|
||||
}
|
||||
|
||||
#saveCapture {
|
||||
display: none;
|
||||
}
|
||||
|
||||
.buttonContainer {
|
||||
display: flex;
|
||||
}
|
||||
|
||||
#status {
|
||||
margin-top: 30px;
|
||||
margin-left: 20px;
|
||||
font-size: 14px;
|
||||
}
|
||||
|
||||
.button {
|
||||
padding: 10px;
|
||||
border: 2px solid darkred;
|
||||
font-size: 16px;
|
||||
background-color: lavenderblush;
|
||||
font-weight: bold;
|
||||
margin: 10px 20px;
|
||||
cursor: pointer;
|
||||
border-radius: 5px;
|
||||
}
|
||||
|
||||
.button:hover {
|
||||
color: red;
|
||||
background-color: darkred;
|
||||
}
|
||||
|
||||
#review {
|
||||
color: blue;
|
||||
display: inline;
|
||||
text-decoration: underline;
|
||||
cursor: pointer;
|
||||
}
|
||||
@@ -1,24 +0,0 @@
|
||||
<html>
|
||||
<head>
|
||||
<title>Audio Transcription Options</title>
|
||||
<script src="complete.js"></script>
|
||||
<link rel="stylesheet" href="complete.css" type="text/css">
|
||||
</head>
|
||||
<body>
|
||||
<div class="header"><img src="./collabora.png"/> <h1 class="header-title">Audio Transcription</h1></div>
|
||||
<div class="inner">
|
||||
<div class="progress">
|
||||
<label for="progrssContainer">Encoding Progress:</label>
|
||||
<div id="progressContainer">
|
||||
<div id="encodeProgress"></div>
|
||||
</div>
|
||||
</div>
|
||||
<p id="status"></p>
|
||||
</div>
|
||||
<div class="buttonContainer">
|
||||
<div class="button" id="saveCapture">Save Capture</div>
|
||||
<div class="button" id="close">Close</div>
|
||||
</div>
|
||||
<div class="notes">Thank you for using the extension! Please go <div id="review">here</div> to leave a star!</div>
|
||||
</body>
|
||||
</html>
|
||||
@@ -1,56 +0,0 @@
|
||||
document.addEventListener('DOMContentLoaded', () => {
|
||||
const encodeProgress = document.getElementById('encodeProgress');
|
||||
const saveButton = document.getElementById('saveCapture');
|
||||
const closeButton = document.getElementById('close');
|
||||
const review = document.getElementById('review');
|
||||
const status = document.getElementById('status');
|
||||
let format;
|
||||
let audioURL;
|
||||
let encoding = false;
|
||||
chrome.runtime.onMessage.addListener((request, sender, sendResponse) => {
|
||||
if(request.type === "createTab") {
|
||||
format = request.format;
|
||||
let startID = request.startID;
|
||||
status.innerHTML = "Please wait..."
|
||||
closeButton.onclick = () => {
|
||||
chrome.runtime.sendMessage({cancelEncodeID: startID});
|
||||
chrome.tabs.getCurrent((tab) => {
|
||||
chrome.tabs.remove(tab.id);
|
||||
});
|
||||
}
|
||||
|
||||
//if the encoding completed before the page has loaded
|
||||
if(request.audioURL) {
|
||||
encodeProgress.style.width = '100%';
|
||||
status.innerHTML = "File is ready!"
|
||||
generateSave(request.audioURL);
|
||||
} else {
|
||||
encoding = true;
|
||||
}
|
||||
}
|
||||
|
||||
//when encoding completes
|
||||
if(request.type === "encodingComplete" && encoding) {
|
||||
encoding = false;
|
||||
status.innerHTML = "File is ready!";
|
||||
encodeProgress.style.width = '100%';
|
||||
generateSave(request.audioURL);
|
||||
}
|
||||
//updates encoding process bar upon messages
|
||||
if(request.type === "encodingProgress" && encoding) {
|
||||
encodeProgress.style.width = `${request.progress * 100}%`;
|
||||
}
|
||||
function generateSave(url) { //creates the save button
|
||||
const currentDate = new Date(Date.now()).toDateString();
|
||||
saveButton.onclick = () => {
|
||||
chrome.downloads.download({url: url, filename: `${currentDate}.${format}`, saveAs: true});
|
||||
};
|
||||
saveButton.style.display = "inline-block";
|
||||
}
|
||||
});
|
||||
review.onclick = () => {
|
||||
chrome.tabs.create({url: "https://github.com/collabora/whisper-live"});
|
||||
}
|
||||
|
||||
|
||||
})
|
||||
@@ -1,3 +1,5 @@
|
||||
|
||||
|
||||
var elem_container = null;
|
||||
var elem_text = null;
|
||||
|
||||
@@ -111,17 +113,33 @@ function get_lines(elem, line_height) {
|
||||
|
||||
}
|
||||
|
||||
chrome.runtime.onMessage.addListener(function(request, sender) {
|
||||
function remove_element() {
|
||||
var elem = document.getElementById('transcription')
|
||||
for (var i = 0; i < 4; i++) {
|
||||
document.getElementById("t" + i).remove();
|
||||
}
|
||||
elem.remove()
|
||||
}
|
||||
|
||||
chrome.runtime.onMessage.addListener((request, sender, sendResponse) => {
|
||||
const { type, data } = request;
|
||||
|
||||
if (type === "STOP") {
|
||||
remove_element();
|
||||
sendResponse({data: "STOPPED"});
|
||||
return;
|
||||
}
|
||||
|
||||
init_element();
|
||||
|
||||
message = JSON.parse(request.message);
|
||||
message = JSON.parse(data);
|
||||
|
||||
var text = '';
|
||||
for (var i = 0; i < message.segments.length; i++) {
|
||||
text += message.segments[i].text + ' ';
|
||||
for (var i = 0; i < message.length; i++) {
|
||||
text += message[i].text + ' ';
|
||||
}
|
||||
text = text.replace(/(\r\n|\n|\r)/gm, "");
|
||||
|
||||
|
||||
var elem = document.getElementById('t3');
|
||||
elem.innerHTML = text;
|
||||
|
||||
@@ -132,6 +150,7 @@ chrome.runtime.onMessage.addListener(function(request, sender) {
|
||||
|
||||
text_segments = [];
|
||||
text_segments = get_lines(elem, line_height);
|
||||
|
||||
elem.innerHTML = '';
|
||||
|
||||
if (text_segments.length > 2) {
|
||||
@@ -160,4 +179,6 @@ chrome.runtime.onMessage.addListener(function(request, sender) {
|
||||
var elem = document.getElementById('t' + i);
|
||||
elem.style.top = parent_elem.offsetHeight + parent_elem.offsetTop + 'px';
|
||||
}
|
||||
});
|
||||
|
||||
sendResponse({});
|
||||
});
|
||||
|
||||
-14
File diff suppressed because one or more lines are too long
Binary file not shown.
@@ -1 +0,0 @@
|
||||
(function(n){var a=Math.min,s=Math.max;var e=function(n,a,e){var s=e.length;for(var t=0;t<s;++t)n.setUint8(a+t,e.charCodeAt(t))};var t=function(t,e){this.sampleRate=t;this.numChannels=e;this.numSamples=0;this.dataViews=[]};t.prototype.encode=function(r){var t=r[0].length,u=this.numChannels,h=new DataView(new ArrayBuffer(t*u*2)),o=0;for(var e=0;e<t;++e)for(var n=0;n<u;++n){var i=r[n][e]*32767;h.setInt16(o,i<0?s(i,-32768):a(i,32767),true);o+=2}this.dataViews.push(h);this.numSamples+=t};t.prototype.finish=function(s){var n=this.numChannels*this.numSamples*2,t=new DataView(new ArrayBuffer(44));e(t,0,"RIFF");t.setUint32(4,36+n,true);e(t,8,"WAVE");e(t,12,"fmt ");t.setUint32(16,16,true);t.setUint16(20,1,true);t.setUint16(22,this.numChannels,true);t.setUint32(24,this.sampleRate,true);t.setUint32(28,this.sampleRate*4,true);t.setUint16(32,this.numChannels*2,true);t.setUint16(34,16,true);e(t,36,"data");t.setUint32(40,n,true);this.dataViews.unshift(t);var a=new Blob(this.dataViews,{type:"audio/wav"});this.cleanup();return a};t.prototype.cancel=t.prototype.cleanup=function(){delete this.dataViews};n.WavAudioEncoder=t})(self);
|
||||
@@ -1,23 +0,0 @@
|
||||
.header {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
padding-bottom: 15px;
|
||||
border-bottom: 2px solid darkred;
|
||||
}
|
||||
|
||||
h1 {
|
||||
font-size: 36px;
|
||||
}
|
||||
|
||||
img {
|
||||
height: 64px;
|
||||
margin: 0 20px 0 0;
|
||||
}
|
||||
|
||||
h2 {
|
||||
font-size: 26px;
|
||||
}
|
||||
|
||||
.inner {
|
||||
text-align: center;
|
||||
}
|
||||
@@ -1,14 +0,0 @@
|
||||
<html>
|
||||
<head>
|
||||
<title>Audio Transcription Options</title>
|
||||
<link rel="stylesheet" href="error.css" type="text/css">
|
||||
</head>
|
||||
<body>
|
||||
<div class="header"><img src="./collabora.png"/> <h1>Audio Transcription</h1></div>
|
||||
<div class="inner">
|
||||
<h2>Sorry, capture on YouTube is disabled!</h2>
|
||||
<p>Chrome Web Store does not allow extensions to capture audio from YouTube due to copyright reasons.</p>
|
||||
<p>Sorry for the inconvenience, please use the extension on other websites.</p>
|
||||
</div>
|
||||
</body>
|
||||
</html>
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 5.5 KiB |
@@ -1,53 +1,25 @@
|
||||
{
|
||||
"manifest_version": 2,
|
||||
{
|
||||
"manifest_version": 3,
|
||||
|
||||
"name": "Audio Transcription",
|
||||
"description": "This extension captures the audio on the current tab and saves the output file on your computer when the capture is complete",
|
||||
"version": "1.1.1",
|
||||
"icons": {
|
||||
"128":"collabora.png"
|
||||
"name": "Audio Transcription",
|
||||
"version": "1.0.0",
|
||||
"description": "This extension captures the audio on the current tab, sends it to a server for transcription and shows the transcription in Real-time.",
|
||||
|
||||
"options_page": "options.html",
|
||||
"background": {
|
||||
"service_worker": "background.js"
|
||||
},
|
||||
|
||||
"browser_action": {
|
||||
"default_icon": "collabora.png",
|
||||
"default_popup": "popup.html",
|
||||
"default_title": "Open Audio Transcription interface"
|
||||
},
|
||||
"options_page": "options.html",
|
||||
"background": {
|
||||
"page" : "background.html",
|
||||
"persistent": true
|
||||
},
|
||||
"content_security_policy": "script-src 'self' https://cdn.jsdelivr.net 'wasm-eval'; object-src 'self'",
|
||||
"permissions": [
|
||||
"tabCapture",
|
||||
"downloads",
|
||||
"storage",
|
||||
"activeTab",
|
||||
"tabs",
|
||||
"*://*/*"
|
||||
],
|
||||
"externally_connectable": {
|
||||
"matches": ["<all_urls>"]
|
||||
},
|
||||
"content_scripts": [{
|
||||
"js": ["content.js"],
|
||||
"matches": ["<all_urls>"]
|
||||
}],
|
||||
"commands": {
|
||||
"start": {
|
||||
"suggested_key": {
|
||||
"default": "Ctrl+Shift+S",
|
||||
"mac": "Command+Shift+U"
|
||||
},
|
||||
"description": "Start Capture"
|
||||
},
|
||||
"stop": {
|
||||
"suggested_key": {
|
||||
"default": "Ctrl+Shift+X",
|
||||
"mac": "MacCtrl+Shift+X"
|
||||
},
|
||||
"description": "Stop Capture"
|
||||
"permissions": [
|
||||
"storage",
|
||||
"activeTab",
|
||||
"tabCapture",
|
||||
"scripting"
|
||||
],
|
||||
"icons": {
|
||||
"128":"collabora.png"
|
||||
},
|
||||
"action": {
|
||||
"default_popup": "popup.html",
|
||||
"default_icon": "collabora.png"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,33 +1,18 @@
|
||||
<html>
|
||||
<head>
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<title>Audio Transcription Options</title>
|
||||
<script src="options.js"></script>
|
||||
<meta http-equiv="X-UA-Compatible" content="IE=edge">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<link rel="stylesheet" href="options.css" type="text/css">
|
||||
</head>
|
||||
<body>
|
||||
</head>
|
||||
|
||||
<body>
|
||||
<div class="header"><img src="./collabora.png"/> <h1>Audio Transcription</h1></div>
|
||||
<div class="inner">
|
||||
<h2>Options</h2>
|
||||
<ul class="options-list">
|
||||
<li><input type="checkbox" id="mute"><label for="mute">Mute tabs that are being captured</label></li>
|
||||
<li class="time"><label for="maxTime">Maximum capture time <p class="limit">(enter value from 1 - 20)</p>: </label><input type="text" id="maxTime"> min(s)</li>
|
||||
<li><input type="checkbox" id="removeLimit"><label for="removeLimit">Remove capture time limit (not recommended)</label></li>
|
||||
<li id="outputType"><label for="outputType">Output file format:</label>
|
||||
<input class="radioChoice" id="mp3" type="radio" name="format" value="mp3"> <label for="mp3">.mp3</label>
|
||||
<input class="radioChoice" id="wav" type="radio" name="format" value="wav"> <label for="wav">.wav</label>
|
||||
</li>
|
||||
<li id="qualityLi">
|
||||
<label for="quality">MP3 Quality: </label>
|
||||
<select id="quality">
|
||||
<option value="96">Low</option>
|
||||
<option value="192">Medium</option>
|
||||
<option value="320">High</option>
|
||||
</select>
|
||||
</li>
|
||||
<li><input type="checkbox" id="doVad"><label for="doVad">Enable Voice Activity detection.</label></li>
|
||||
</ul>
|
||||
</div>
|
||||
<div class="button" id="save">Save Settings</div>
|
||||
<div id="status"></div>
|
||||
</body>
|
||||
</html>
|
||||
<div class="button" id="stop">Stop Capture</div>
|
||||
</body>
|
||||
|
||||
</html>
|
||||
@@ -1,100 +1,100 @@
|
||||
document.addEventListener('DOMContentLoaded', () => {
|
||||
const mute = document.getElementById('mute');
|
||||
const maxTime = document.getElementById('maxTime');
|
||||
const save = document.getElementById('save');
|
||||
const status = document.getElementById('status');
|
||||
const mp3Select = document.getElementById('mp3');
|
||||
const wavSelect = document.getElementById('wav');
|
||||
const quality = document.getElementById("quality");
|
||||
const qualityLi = document.getElementById("qualityLi");
|
||||
const limitRemoved = document.getElementById("removeLimit");
|
||||
const doVad = document.getElementById("doVad");
|
||||
let currentFormat;
|
||||
//initial settings
|
||||
chrome.storage.sync.get({
|
||||
muteTab: false,
|
||||
maxTime: 1200000,
|
||||
format: "mp3",
|
||||
quality: 192,
|
||||
limitRemoved: false,
|
||||
asr: false,
|
||||
doVad: false
|
||||
}, (options) => {
|
||||
mute.checked = options.muteTab;
|
||||
limitRemoved.checked = options.limitRemoved;
|
||||
maxTime.disabled = options.limitRemoved;
|
||||
maxTime.value = options.maxTime/60000;
|
||||
currentFormat = options.format;
|
||||
doVad.checked = options.doVad;
|
||||
if (options.format === "mp3") {
|
||||
mp3Select.checked = true;
|
||||
qualityLi.style.display = "block";
|
||||
} else {
|
||||
wavSelect.checked = true;
|
||||
}
|
||||
if (options.quality === "96") {
|
||||
quality.selectedIndex = 0;
|
||||
} else if(options.quality === "192") {
|
||||
quality.selectedIndex = 1;
|
||||
} else {
|
||||
quality.selectedIndex = 2;
|
||||
}
|
||||
/**
|
||||
* Captures audio from the active tab in Google Chrome.
|
||||
* @returns {Promise<MediaStream>} A promise that resolves with the captured audio stream.
|
||||
*/
|
||||
function captureTabAudio() {
|
||||
return new Promise((resolve) => {
|
||||
chrome.tabCapture.capture(
|
||||
{
|
||||
audio: true,
|
||||
video: false,
|
||||
},
|
||||
(stream) => {
|
||||
resolve(stream);
|
||||
}
|
||||
);
|
||||
});
|
||||
}
|
||||
|
||||
mute.onchange = () => {
|
||||
status.innerHTML = "";
|
||||
}
|
||||
|
||||
doVad.onchange = () => {
|
||||
status.innerHTML = "";
|
||||
}
|
||||
|
||||
maxTime.onchange = () => {
|
||||
status.innerHTML = "";
|
||||
if(maxTime.value > 20) {
|
||||
maxTime.value = 20;
|
||||
} else if (maxTime.value < 1) {
|
||||
maxTime.value = 1;
|
||||
} else if (isNaN(maxTime.value)) {
|
||||
maxTime.value = 20;
|
||||
}
|
||||
}
|
||||
|
||||
mp3Select.onclick = () => {
|
||||
currentFormat = "mp3";
|
||||
qualityLi.style.display = "block";
|
||||
status.innerHTML = "";
|
||||
}
|
||||
|
||||
wavSelect.onclick = () => {
|
||||
currentFormat = "wav";
|
||||
qualityLi.style.display = "none";
|
||||
status.innerHTML = "";
|
||||
}
|
||||
|
||||
quality.onchange = (e) => {
|
||||
status.innerHTML = "";
|
||||
}
|
||||
|
||||
limitRemoved.onchange = () => {
|
||||
if(limitRemoved.checked) {
|
||||
maxTime.disabled = true;
|
||||
status.innerHTML = "WARNING: Recordings that are too long may not save properly!"
|
||||
} else {
|
||||
maxTime.disabled = false;
|
||||
status.innerHTML = "";
|
||||
}
|
||||
}
|
||||
|
||||
save.onclick = () => {
|
||||
chrome.storage.sync.set({
|
||||
muteTab: mute.checked,
|
||||
maxTime: maxTime.value*60000,
|
||||
format: currentFormat,
|
||||
quality: quality.value,
|
||||
limitRemoved: limitRemoved.checked,
|
||||
doVad: doVad.checked
|
||||
/**
|
||||
* Sends a message to a specific tab in Google Chrome.
|
||||
* @param {number} tabId - The ID of the tab to send the message to.
|
||||
* @param {any} data - The data to be sent as the message.
|
||||
* @returns {Promise<any>} A promise that resolves with the response from the tab.
|
||||
*/
|
||||
function sendMessageToTab(tabId, data) {
|
||||
return new Promise((resolve) => {
|
||||
chrome.tabs.sendMessage(tabId, data, (response) => {
|
||||
resolve(response);
|
||||
});
|
||||
status.innerHTML = "Settings saved!"
|
||||
});
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* Starts recording audio from the captured tab.
|
||||
* @param {Object} option - The options object containing the currentTabId.
|
||||
*/
|
||||
async function startRecord(option) {
|
||||
const stream = await captureTabAudio();
|
||||
if (stream) {
|
||||
// call when the stream inactive
|
||||
stream.oninactive = () => {
|
||||
window.close();
|
||||
};
|
||||
|
||||
const socket = new WebSocket("ws://localhost:9090/");
|
||||
socket.onopen = function(e) {
|
||||
socket.send("handshake");
|
||||
};
|
||||
|
||||
socket.onmessage = async (event) => {
|
||||
// console.log(event.data);
|
||||
await sendMessageToTab(option.currentTabId, {
|
||||
data: event.data,
|
||||
});
|
||||
};
|
||||
|
||||
const audioDataCache = [];
|
||||
const context = new AudioContext({sampleRate: 16000});
|
||||
const mediaStream = context.createMediaStreamSource(stream);
|
||||
const recorder = context.createScriptProcessor(4096, 1, 1);
|
||||
|
||||
recorder.onaudioprocess = async (event) => {
|
||||
if (!context) return;
|
||||
|
||||
const inputData = event.inputBuffer.getChannelData(0);
|
||||
|
||||
audioDataCache.push(inputData);
|
||||
socket.send(inputData);
|
||||
};
|
||||
|
||||
// Prevent page mute
|
||||
mediaStream.connect(recorder);
|
||||
recorder.connect(context.destination);
|
||||
mediaStream.connect(context.destination);
|
||||
} else {
|
||||
window.close();
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Listener for incoming messages from the extension's background script.
|
||||
* @param {Object} request - The message request object.
|
||||
* @param {Object} sender - The sender object containing information about the message sender.
|
||||
* @param {Function} sendResponse - The function to send a response back to the message sender.
|
||||
*/
|
||||
chrome.runtime.onMessage.addListener(async (request, sender, sendResponse) => {
|
||||
const { type, data } = request;
|
||||
|
||||
switch (type) {
|
||||
case "start_capture":
|
||||
await startRecord(data);
|
||||
break;
|
||||
default:
|
||||
break;
|
||||
}
|
||||
|
||||
sendResponse({});
|
||||
});
|
||||
|
||||
@@ -1,89 +0,0 @@
|
||||
body, h2 {
|
||||
font-family: Ubuntu, sans-serif;
|
||||
}
|
||||
|
||||
body {
|
||||
margin: 10px 15px 5px 15px;
|
||||
}
|
||||
|
||||
#status, #timeRem {
|
||||
margin: 10px auto;
|
||||
text-align: center;
|
||||
font-size: 20px;
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
.header {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
padding-bottom: 15px;
|
||||
border-bottom: 2px solid darkred;
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
.header img {
|
||||
height: 64px;
|
||||
margin: 0 20px 0 0;
|
||||
}
|
||||
|
||||
ul {
|
||||
font-size: 16px;
|
||||
font-weight: bold;
|
||||
list-style: none;
|
||||
padding: 0;
|
||||
}
|
||||
|
||||
li {
|
||||
font-size: 14px;
|
||||
font-weight: normal;
|
||||
white-space: nowrap;
|
||||
margin: 0 0 0 5px;
|
||||
}
|
||||
|
||||
.extra {
|
||||
font-family: sans-serif;
|
||||
white-space: nowrap;
|
||||
font-style: italic;
|
||||
margin: 3px;
|
||||
}
|
||||
|
||||
.links {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: space-between;
|
||||
font-family: sans-serif;
|
||||
}
|
||||
|
||||
.links p {
|
||||
color: blue;
|
||||
text-decoration: underline;
|
||||
cursor: pointer;
|
||||
}
|
||||
|
||||
.buttonContainer {
|
||||
display: flex;
|
||||
justify-content: space-around;
|
||||
}
|
||||
|
||||
.button {
|
||||
display: none;
|
||||
text-align: center;
|
||||
padding: 10px;
|
||||
border: 2px solid darkred;
|
||||
font-size: 16px;
|
||||
font-weight: bold;
|
||||
cursor: pointer;
|
||||
background-color: lavenderblush;
|
||||
border-radius: 5px;
|
||||
}
|
||||
|
||||
.button:hover {
|
||||
color: red;
|
||||
background-color: darkred;
|
||||
}
|
||||
|
||||
.notes {
|
||||
padding: 10px;
|
||||
font-size: 12px;
|
||||
white-space: pre;
|
||||
}
|
||||
@@ -1,29 +1,15 @@
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<title>Audio Transcription</title>
|
||||
<script src="popup.js"></script>
|
||||
<link rel="stylesheet" href="popup.css" type="text/css">
|
||||
</head>
|
||||
<body>
|
||||
<div class="header"><img src="./collabora.png"/> <h1>Audio Transcription</h1></div>
|
||||
<div id="status"></div>
|
||||
<div id="timeRem"></div>
|
||||
<div class="buttonContainer">
|
||||
<div class="button" id="start">Start Capture</div>
|
||||
<div class="button" id="finish">Save Capture</div>
|
||||
<div class="button" id="cancel">Cancel Capture</div>
|
||||
</div>
|
||||
<div class="notes">After capture is finished, a new tab will be opened automatically for you to
|
||||
name and save the file. Please do not close the tab before saving the file!</div>
|
||||
<ul> Hotkeys:
|
||||
<li id="startKey">Ctrl/Command + Shift + to start capture on current tab</li>
|
||||
<li id="endKey">Ctrl/Command + Shift + X to stop capture on current tab</li>
|
||||
</ul>
|
||||
<p class="extra">Hotkeys may not work if another extension is using the same hotkeys</p>
|
||||
<p class="extra">Currently the max capture time is 20 minutes due to Chrome memory contraints</p>
|
||||
<div class="links">
|
||||
<p id="options">Options</p>
|
||||
<p id="GitHub">GitHub</p>
|
||||
</div>
|
||||
</body>
|
||||
</html>
|
||||
<head>
|
||||
<title>Audio Capture</title>
|
||||
<script src="popup.js"></script>
|
||||
<link rel="stylesheet" href="style.css" type="text/css">
|
||||
</head>
|
||||
<body>
|
||||
<div class="header"><img src="./collabora.png"/> <h1>Audio Transcription</h1></div>
|
||||
<div class="button-container">
|
||||
<div class="button" id="startCapture">Start Capture</div>
|
||||
<div class="button" id="stopCapture" disabled>Stop Capture</div>
|
||||
</div>
|
||||
</body>
|
||||
</html>
|
||||
+65
-140
@@ -1,145 +1,70 @@
|
||||
let interval;
|
||||
let timeLeft;
|
||||
// Wait for the DOM content to be fully loaded
|
||||
document.addEventListener("DOMContentLoaded", function () {
|
||||
const startButton = document.getElementById("startCapture");
|
||||
const stopButton = document.getElementById("stopCapture");
|
||||
|
||||
const displayStatus = function() { //function to handle the display of time and buttons
|
||||
chrome.tabs.query({active: true, currentWindow: true}, (tabs) => {
|
||||
const status = document.getElementById("status");
|
||||
const timeRem = document.getElementById("timeRem");
|
||||
const startButton = document.getElementById('start');
|
||||
const finishButton = document.getElementById('finish');
|
||||
const cancelButton = document.getElementById('cancel');
|
||||
//CODE TO BLOCK CAPTURE ON YOUTUBE, DO NOT DELETE
|
||||
// if(tabs[0].url.toLowerCase().includes("youtube")) {
|
||||
// status.innerHTML = "Capture is disabled on this site due to copyright";
|
||||
// } else {
|
||||
chrome.runtime.sendMessage({currentTab: tabs[0].id}, (response) => {
|
||||
if(response) {
|
||||
chrome.storage.sync.get({
|
||||
maxTime: 1200000,
|
||||
limitRemoved: false
|
||||
}, (options) => {
|
||||
if(options.maxTime > 1200000) {
|
||||
chrome.storage.sync.set({
|
||||
maxTime: 1200000
|
||||
});
|
||||
timeLeft = 1200000 - (Date.now() - response)
|
||||
} else {
|
||||
timeLeft = options.maxTime - (Date.now() - response)
|
||||
}
|
||||
status.innerHTML = "Tab is currently being captured";
|
||||
if(options.limitRemoved) {
|
||||
timeRem.innerHTML = `${parseTime(Date.now() - response)}`;
|
||||
interval = setInterval(() => {
|
||||
timeRem.innerHTML = `${parseTime(Date.now() - response)}`;
|
||||
});
|
||||
} else {
|
||||
timeRem.innerHTML = `${parseTime(timeLeft)} remaining`;
|
||||
interval = setInterval(() => {
|
||||
timeLeft = timeLeft - 1000;
|
||||
timeRem.innerHTML = `${parseTime(timeLeft)} remaining`;
|
||||
}, 1000);
|
||||
}
|
||||
});
|
||||
finishButton.style.display = "block";
|
||||
cancelButton.style.display = "block";
|
||||
} else {
|
||||
startButton.style.display = "block";
|
||||
}
|
||||
});
|
||||
// }
|
||||
});
|
||||
}
|
||||
// Add click event listeners to the buttons
|
||||
startButton.addEventListener("click", startCapture);
|
||||
stopButton.addEventListener("click", stopCapture);
|
||||
|
||||
const parseTime = function(time) { //function to display time remaining or time elapsed
|
||||
let minutes = Math.floor((time/1000)/60);
|
||||
let seconds = Math.floor((time/1000) % 60);
|
||||
if (minutes < 10 && minutes >= 0) {
|
||||
minutes = '0' + minutes;
|
||||
} else if (minutes < 0) {
|
||||
minutes = '00';
|
||||
}
|
||||
if (seconds < 10 && seconds >= 0) {
|
||||
seconds = '0' + seconds;
|
||||
} else if (seconds < 0) {
|
||||
seconds = '00';
|
||||
}
|
||||
return `${minutes}:${seconds}`
|
||||
}
|
||||
|
||||
//manipulation of the displayed buttons upon message from background
|
||||
chrome.runtime.onMessage.addListener((request, sender) => {
|
||||
chrome.tabs.query({active: true, currentWindow: true}, (tabs) => {
|
||||
const status = document.getElementById("status");
|
||||
const timeRem = document.getElementById("timeRem");
|
||||
const buttons = document.getElementById("buttons");
|
||||
const startButton = document.getElementById('start');
|
||||
const finishButton = document.getElementById('finish');
|
||||
const cancelButton = document.getElementById('cancel');
|
||||
if(request.captureStarted && request.captureStarted === tabs[0].id) {
|
||||
chrome.storage.sync.get({
|
||||
maxTime: 1200000,
|
||||
limitRemoved: false
|
||||
}, (options) => {
|
||||
if(options.maxTime > 1200000) {
|
||||
chrome.storage.sync.set({
|
||||
maxTime: 1200000
|
||||
});
|
||||
timeLeft = 1200000 - (Date.now() - request.startTime)
|
||||
} else {
|
||||
timeLeft = options.maxTime - (Date.now() - request.startTime)
|
||||
}
|
||||
status.innerHTML = "Tab is currently being captured";
|
||||
if(options.limitRemoved) {
|
||||
timeRem.innerHTML = `${parseTime(Date.now() - request.startTime)}`;
|
||||
interval = setInterval(() => {
|
||||
timeRem.innerHTML = `${parseTime(Date.now() - request.startTime)}`
|
||||
}, 1000);
|
||||
} else {
|
||||
timeRem.innerHTML = `${parseTime(timeLeft)} remaining`;
|
||||
interval = setInterval(() => {
|
||||
timeLeft = timeLeft - 1000;
|
||||
timeRem.innerHTML = `${parseTime(timeLeft)} remaining`;
|
||||
}, 1000);
|
||||
}
|
||||
});
|
||||
finishButton.style.display = "block";
|
||||
cancelButton.style.display = "block";
|
||||
startButton.style.display = "none";
|
||||
} else if(request.captureStopped && request.captureStopped === tabs[0].id) {
|
||||
status.innerHTML = "";
|
||||
finishButton.style.display = "none";
|
||||
cancelButton.style.display = "none";
|
||||
startButton.style.display = "block";
|
||||
timeRem.innerHTML = "";
|
||||
clearInterval(interval);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
//initial display for popup menu when opened
|
||||
document.addEventListener('DOMContentLoaded', function() {
|
||||
displayStatus();
|
||||
const startKey = document.getElementById("startKey");
|
||||
const endKey = document.getElementById("endKey");
|
||||
const startButton = document.getElementById('start');
|
||||
const finishButton = document.getElementById('finish');
|
||||
const cancelButton = document.getElementById('cancel');
|
||||
startButton.onclick = () => {chrome.runtime.sendMessage("startCapture")};
|
||||
finishButton.onclick = () => {chrome.runtime.sendMessage("stopCapture")};
|
||||
cancelButton.onclick = () => {chrome.runtime.sendMessage("cancelCapture")};
|
||||
chrome.runtime.getPlatformInfo((info) => {
|
||||
if(info.os === "mac") {
|
||||
startKey.innerHTML = "Command + Shift + U to start capture on current tab";
|
||||
endKey.innerHTML = "Command + Shift + X to stop capture on current tab";
|
||||
// Retrieve capturing state from storage on popup open
|
||||
chrome.storage.local.get("capturingState", ({ capturingState }) => {
|
||||
if (capturingState && capturingState.isCapturing) {
|
||||
toggleCaptureButtons(true);
|
||||
} else {
|
||||
startKey.innerHTML = "Ctrl + Shift + S to start capture on current tab";
|
||||
endKey.innerHTML = "Ctrl + Shift + X to stop capture on current tab";
|
||||
toggleCaptureButtons(false);
|
||||
}
|
||||
})
|
||||
const options = document.getElementById("options");
|
||||
options.onclick = () => {chrome.runtime.openOptionsPage()};
|
||||
const git = document.getElementById("GitHub");
|
||||
git.onclick = () => {chrome.tabs.create({url: "https://github.com/arblast/Chrome-Audio-Capturer"})};
|
||||
});
|
||||
|
||||
});
|
||||
// Function to handle the start capture button click event
|
||||
async function startCapture() {
|
||||
// Ignore click if the button is disabled
|
||||
if (startButton.disabled) {
|
||||
return;
|
||||
}
|
||||
|
||||
// Get the current active tab
|
||||
const currentTab = await getCurrentTab();
|
||||
|
||||
// Send a message to the background script to start capturing
|
||||
chrome.runtime.sendMessage({ action: "startCapture", tabId: currentTab.id }, () => {
|
||||
// Update capturing state in storage and toggle the buttons
|
||||
chrome.storage.local.set({ capturingState: { isCapturing: true } }, () => {
|
||||
toggleCaptureButtons(true);
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
// Function to handle the stop capture button click event
|
||||
function stopCapture() {
|
||||
// Ignore click if the button is disabled
|
||||
if (stopButton.disabled) {
|
||||
return;
|
||||
}
|
||||
|
||||
// Send a message to the background script to stop capturing
|
||||
chrome.runtime.sendMessage({ action: "stopCapture" }, () => {
|
||||
// Update capturing state in storage and toggle the buttons
|
||||
chrome.storage.local.set({ capturingState: { isCapturing: false } }, () => {
|
||||
toggleCaptureButtons(false);
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
// Function to get the current active tab
|
||||
async function getCurrentTab() {
|
||||
return new Promise((resolve) => {
|
||||
chrome.tabs.query({ active: true, currentWindow: true }, (tabs) => {
|
||||
resolve(tabs[0]);
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
// Function to toggle the capture buttons based on the capturing state
|
||||
function toggleCaptureButtons(isCapturing) {
|
||||
startButton.disabled = isCapturing;
|
||||
stopButton.disabled = !isCapturing;
|
||||
startButton.classList.toggle("disabled", isCapturing);
|
||||
stopButton.classList.toggle("disabled", !isCapturing);
|
||||
}
|
||||
});
|
||||
Binary file not shown.
@@ -54,20 +54,29 @@ label {
|
||||
margin-left: 15px;
|
||||
}
|
||||
|
||||
.button-container {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
padding: 10px;
|
||||
}
|
||||
|
||||
.button {
|
||||
padding: 10px;
|
||||
border: 2px solid darkred;
|
||||
font-size: 16px;
|
||||
background-color: lavenderblush;
|
||||
font-weight: bold;
|
||||
margin: 10px 20px;
|
||||
cursor: pointer;
|
||||
white-space: nowrap;
|
||||
width: 110px;
|
||||
width: 150px;
|
||||
border-radius: 5px;
|
||||
}
|
||||
|
||||
.button:hover {
|
||||
.disabled {
|
||||
opacity: 0.6;
|
||||
cursor: not-allowed;
|
||||
}
|
||||
|
||||
.button:hover:not(:disabled) {
|
||||
color: red;
|
||||
background-color: darkred;
|
||||
}
|
||||
@@ -1,148 +0,0 @@
|
||||
let recLength = 0,
|
||||
recBuffers = [],
|
||||
sampleRate,
|
||||
numChannels;
|
||||
|
||||
onmessage = function(e) {
|
||||
switch (e.data.command) {
|
||||
case 'init':
|
||||
init(e.data.config);
|
||||
break;
|
||||
case 'record':
|
||||
record(e.data.buffer);
|
||||
break;
|
||||
case 'exportWAV':
|
||||
exportWAV(e.data.type);
|
||||
break;
|
||||
case 'getBuffer':
|
||||
getBuffer();
|
||||
break;
|
||||
case 'clear':
|
||||
clear();
|
||||
break;
|
||||
}
|
||||
};
|
||||
|
||||
function init(config) {
|
||||
sampleRate = config.sampleRate;
|
||||
numChannels = config.numChannels;
|
||||
initBuffers();
|
||||
}
|
||||
|
||||
function record(inputBuffer) {
|
||||
for (var channel = 0; channel < numChannels; channel++) {
|
||||
recBuffers[channel].push(inputBuffer[channel]);
|
||||
}
|
||||
recLength += inputBuffer[0].length;
|
||||
}
|
||||
|
||||
function exportWAV(type) {
|
||||
let buffers = [];
|
||||
for (let channel = 0; channel < numChannels; channel++) {
|
||||
buffers.push(mergeBuffers(recBuffers[channel], recLength));
|
||||
}
|
||||
let interleaved;
|
||||
if (numChannels === 2) {
|
||||
interleaved = interleave(buffers[0], buffers[1]);
|
||||
} else {
|
||||
interleaved = buffers[0];
|
||||
}
|
||||
let dataview = encodeWAV(interleaved);
|
||||
let audioBlob = new Blob([dataview], {type: type});
|
||||
|
||||
this.postMessage({command: 'exportWAV', data: audioBlob});
|
||||
}
|
||||
|
||||
function getBuffer() {
|
||||
let buffers = [];
|
||||
for (let channel = 0; channel < numChannels; channel++) {
|
||||
buffers.push(mergeBuffers(recBuffers[channel], recLength));
|
||||
}
|
||||
this.postMessage({command: 'getBuffer', data: buffers});
|
||||
}
|
||||
|
||||
function clear() {
|
||||
recLength = 0;
|
||||
recBuffers = [];
|
||||
initBuffers();
|
||||
}
|
||||
|
||||
function initBuffers() {
|
||||
for (let channel = 0; channel < numChannels; channel++) {
|
||||
recBuffers[channel] = [];
|
||||
}
|
||||
}
|
||||
|
||||
function mergeBuffers(recBuffers, recLength) {
|
||||
let result = new Float32Array(recLength);
|
||||
let offset = 0;
|
||||
for (let i = 0; i < recBuffers.length; i++) {
|
||||
result.set(recBuffers[i], offset);
|
||||
offset += recBuffers[i].length;
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
function interleave(inputL, inputR) {
|
||||
let length = inputL.length + inputR.length;
|
||||
let result = new Float32Array(length);
|
||||
|
||||
let index = 0,
|
||||
inputIndex = 0;
|
||||
|
||||
while (index < length) {
|
||||
result[index++] = inputL[inputIndex];
|
||||
result[index++] = inputR[inputIndex];
|
||||
inputIndex++;
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
function floatTo16BitPCM(output, offset, input) {
|
||||
for (let i = 0; i < input.length; i++, offset += 2) {
|
||||
let s = Math.max(-1, Math.min(1, input[i]));
|
||||
output.setInt16(offset, s < 0 ? s * 0x8000 : s * 0x7FFF, true);
|
||||
}
|
||||
}
|
||||
|
||||
function writeString(view, offset, string) {
|
||||
for (let i = 0; i < string.length; i++) {
|
||||
view.setUint8(offset + i, string.charCodeAt(i));
|
||||
}
|
||||
}
|
||||
|
||||
function encodeWAV(samples) {
|
||||
let buffer = new ArrayBuffer(44 + samples.length * 2);
|
||||
let view = new DataView(buffer);
|
||||
|
||||
/* RIFF identifier */
|
||||
writeString(view, 0, 'RIFF');
|
||||
/* RIFF chunk length */
|
||||
view.setUint32(4, 36 + samples.length * 2, true);
|
||||
/* RIFF type */
|
||||
writeString(view, 8, 'WAVE');
|
||||
/* format chunk identifier */
|
||||
writeString(view, 12, 'fmt ');
|
||||
/* format chunk length */
|
||||
view.setUint32(16, 16, true);
|
||||
/* sample format (raw) */
|
||||
view.setUint16(20, 1, true);
|
||||
/* channel count */
|
||||
view.setUint16(22, numChannels, true);
|
||||
/* sample rate */
|
||||
view.setUint32(24, sampleRate, true);
|
||||
/* byte rate (sample rate * block align) */
|
||||
view.setUint32(28, sampleRate * 4, true);
|
||||
/* block align (channel count * bytes per sample) */
|
||||
view.setUint16(32, numChannels * 2, true);
|
||||
/* bits per sample */
|
||||
view.setUint16(34, 16, true);
|
||||
/* data chunk identifier */
|
||||
writeString(view, 36, 'data');
|
||||
/* data chunk length */
|
||||
view.setUint32(40, samples.length * 2, true);
|
||||
|
||||
floatTo16BitPCM(view, 44, samples);
|
||||
|
||||
return view;
|
||||
}
|
||||
@@ -1,106 +0,0 @@
|
||||
importScripts("/../encoders/Mp3Encoder.min.js");
|
||||
|
||||
let NUM_CH = 2, // constant
|
||||
sampleRate = 16000,
|
||||
options = undefined,
|
||||
maxBuffers = undefined,
|
||||
encoder = undefined,
|
||||
recBuffers = undefined,
|
||||
socket = undefined,
|
||||
bufferCount = 0;
|
||||
|
||||
function error(message) {
|
||||
self.postMessage({ command: "error", message: "mp3: " + message });
|
||||
}
|
||||
|
||||
function init(data) {
|
||||
if (data.config.numChannels === NUM_CH) {
|
||||
sampleRate = data.config.sampleRate;
|
||||
options = data.options;
|
||||
} else
|
||||
error("numChannels must be " + NUM_CH);
|
||||
};
|
||||
|
||||
function setOptions(opt) {
|
||||
if (encoder || recBuffers)
|
||||
error("cannot set options during recording");
|
||||
else
|
||||
options = opt;
|
||||
}
|
||||
|
||||
function start(bufferSize) {
|
||||
maxBuffers = Math.ceil(options.timeLimit * sampleRate / bufferSize);
|
||||
// TODO: get server address from user
|
||||
socket = new WebSocket("ws://localhost:9090/");
|
||||
socket.onopen = function(e) {
|
||||
socket.send("handshake");
|
||||
};
|
||||
socket.onmessage = (event) => {
|
||||
self.postMessage({ command: "transcription", text: event.data});
|
||||
};
|
||||
|
||||
if (options.encodeAfterRecord)
|
||||
recBuffers = [];
|
||||
else
|
||||
encoder = new Mp3LameEncoder(sampleRate, options.mp3.bitRate);
|
||||
}
|
||||
|
||||
function record(buffer) {
|
||||
if (bufferCount++ < maxBuffers){
|
||||
if (encoder)
|
||||
encoder.encode(buffer);
|
||||
else
|
||||
recBuffers.push(buffer);
|
||||
|
||||
// send buffer to server
|
||||
socket.send(buffer[0]);
|
||||
}
|
||||
else
|
||||
self.postMessage({ command: "timeout" });
|
||||
};
|
||||
|
||||
function postProgress(progress) {
|
||||
self.postMessage({ command: "progress", progress: progress });
|
||||
};
|
||||
|
||||
function finish() {
|
||||
if (recBuffers) {
|
||||
postProgress(0);
|
||||
encoder = new Mp3LameEncoder(sampleRate, options.mp3.bitRate);
|
||||
let timeout = Date.now() + options.progressInterval;
|
||||
while (recBuffers.length > 0) {
|
||||
encoder.encode(recBuffers.shift());
|
||||
let now = Date.now();
|
||||
if (now > timeout) {
|
||||
postProgress((bufferCount - recBuffers.length) / bufferCount);
|
||||
timeout = now + options.progressInterval;
|
||||
}
|
||||
}
|
||||
postProgress(1);
|
||||
}
|
||||
self.postMessage({
|
||||
command: "complete",
|
||||
blob: encoder.finish(options.mp3.mimeType)
|
||||
});
|
||||
cleanup();
|
||||
};
|
||||
|
||||
function cleanup() {
|
||||
encoder = recBuffers = undefined;
|
||||
bufferCount = 0;
|
||||
socket.close();
|
||||
}
|
||||
|
||||
self.onmessage = function(event) {
|
||||
let data = event.data;
|
||||
switch (data.command) {
|
||||
case "init": init(data); break;
|
||||
case "options": setOptions(data.options); break;
|
||||
case "start": start(data.bufferSize); break;
|
||||
case "record": record(data.buffer); break;
|
||||
case "finish": finish(); break;
|
||||
case "cancel": cleanup();
|
||||
}
|
||||
};
|
||||
|
||||
self.postMessage({ command: "loaded" });
|
||||
@@ -1,105 +0,0 @@
|
||||
importScripts("/../encoders/WavEncoder.min.js");
|
||||
|
||||
|
||||
let sampleRate = 16000,
|
||||
numChannels = 2,
|
||||
options = undefined,
|
||||
maxBuffers = undefined,
|
||||
encoder = undefined,
|
||||
recBuffers = undefined,
|
||||
socket = undefined,
|
||||
bufferCount = 0;
|
||||
|
||||
function error(message) {
|
||||
self.postMessage({ command: "error", message: "wav: " + message });
|
||||
}
|
||||
|
||||
function init(data) {
|
||||
sampleRate = data.config.sampleRate;
|
||||
numChannels = data.config.numChannels;
|
||||
options = data.options;
|
||||
};
|
||||
|
||||
function setOptions(opt) {
|
||||
if (encoder || recBuffers)
|
||||
error("cannot set options during recording");
|
||||
else
|
||||
options = opt;
|
||||
}
|
||||
|
||||
function start(bufferSize) {
|
||||
maxBuffers = Math.ceil(options.timeLimit * sampleRate / bufferSize);
|
||||
// TODO: get server address from user
|
||||
socket = new WebSocket("ws://localhost:9090/");
|
||||
socket.onopen = function(e) {
|
||||
socket.send("handshake");
|
||||
};
|
||||
socket.onmessage = (event) => {
|
||||
self.postMessage({ command: "transcription", text: event.data});
|
||||
};
|
||||
|
||||
if (options.encodeAfterRecord)
|
||||
recBuffers = [];
|
||||
else
|
||||
encoder = new WavAudioEncoder(sampleRate, numChannels);
|
||||
}
|
||||
|
||||
function record(buffer) {
|
||||
if (bufferCount++ < maxBuffers){
|
||||
if (encoder)
|
||||
encoder.encode(buffer);
|
||||
else if(recBuffers)
|
||||
recBuffers.push(buffer);
|
||||
|
||||
// send buffer to server
|
||||
socket.send(buffer[0]);
|
||||
}
|
||||
else
|
||||
self.postMessage({ command: "timeout" });
|
||||
};
|
||||
|
||||
function postProgress(progress) {
|
||||
self.postMessage({ command: "progress", progress: progress });
|
||||
};
|
||||
|
||||
function finish() {
|
||||
if (recBuffers) {
|
||||
postProgress(0);
|
||||
encoder = new WavAudioEncoder(sampleRate, numChannels);
|
||||
var timeout = Date.now() + options.progressInterval;
|
||||
while (recBuffers.length > 0) {
|
||||
encoder.encode(recBuffers.shift());
|
||||
var now = Date.now();
|
||||
if (now > timeout) {
|
||||
postProgress((bufferCount - recBuffers.length) / bufferCount);
|
||||
timeout = now + options.progressInterval;
|
||||
}
|
||||
}
|
||||
postProgress(1);
|
||||
}
|
||||
self.postMessage({
|
||||
command: "complete",
|
||||
blob: encoder.finish(options.wav.mimeType)
|
||||
});
|
||||
cleanup();
|
||||
};
|
||||
|
||||
function cleanup() {
|
||||
encoder = recBuffers = undefined;
|
||||
bufferCount = 0;
|
||||
socket.close();
|
||||
}
|
||||
|
||||
self.onmessage = function(event) {
|
||||
var data = event.data;
|
||||
switch (data.command) {
|
||||
case "init": init(data); break;
|
||||
case "options": setOptions(data.options); break;
|
||||
case "start": start(data.bufferSize); break;
|
||||
case "record": record(data.buffer); break;
|
||||
case "finish": finish(); break;
|
||||
case "cancel": cleanup();
|
||||
}
|
||||
};
|
||||
|
||||
self.postMessage({ command: "loaded" });
|
||||
Reference in New Issue
Block a user