const regex = /(?=^(\((?:\w+\h*|(?1)\h*)*+\))$)(\()|(?<!^)\G(?:(\w+)(?=\h*\))|(\w+)|\h*|(\()|(\))(?=\h*[\w(])|(\)))/gm;
// Alternative syntax using RegExp constructor
// const regex = new RegExp('(?=^(\\((?:\\w+\\h*|(?1)\\h*)*+\\))$)(\\()|(?<!^)\\G(?:(\\w+)(?=\\h*\\))|(\\w+)|\\h*|(\\()|(\\))(?=\\h*[\\w(])|(\\)))', 'gm')
const str = `(10 AND 20 AND (30 OR (40 AND 50)))
Into
['10', 'AND', '20', 'AND', ['30', 'OR', ['40', 'AND', '50']]]`;
const subst = `${2:+[}${3:+'$3'}${4:+'$4', }${5:+[}${6:+], }${7:+]}`;
// The substituted value will be contained in the result variable
const result = str.replace(regex, subst);
console.log('Substitution result: ', result);
Please keep in mind that these code samples are automatically generated and are not guaranteed to work. If you find any syntax errors, feel free to submit a bug report. For a full regex reference for JavaScript, please visit: https://developer.mozilla.org/en/docs/Web/JavaScript/Guide/Regular_Expressions