To trigger JavaScript from InDesign, create a new Hyperlink using the Hyperlinks panel (Window > Interactive).

Select URL from the Link To menu.

In the Destination, start with "javascript:" and then type the JavaScript code that you wish to execute.

indesign hyperlinks panel, adding a URL with JS

In the example above, the code will trigger an alert in the browser with "test" as the text in the alert.

(alert is a built-in JavaScript. It's simply used for demonstration purposes in this article).

You can apply the same type of code as a button action via the Buttons & Forms panel (Window > Interactive).

To do so, add a Go To URL action to your button.

adding a go to URL action

Then type "javascript:" and the code you'd like to execute.

go to url with JS code

You can attach a JavaScript file with custom functions in the Resources section of the in5 export dialog.

When you export with in5 (in5 > Export HTML5 with in5...), you'll send this in action. 

working alert in the browser