Skip to content

Added attention visualization function#202

Merged
kan-bayashi merged 18 commits intoespnet:masterfrom
kan-bayashi:att_vis
Jun 11, 2018
Merged

Added attention visualization function#202
kan-bayashi merged 18 commits intoespnet:masterfrom
kan-bayashi:att_vis

Conversation

@kan-bayashi
Copy link
Copy Markdown
Member

@kan-bayashi kan-bayashi commented Jun 5, 2018 •
edited
Loading

Added attention visualization function (#172).

@kan-bayashi kan-bayashi requested a review from sw005320 June 5, 2018 06:08
@kan-bayashi
Copy link
Copy Markdown
Member Author

kan-bayashi commented Jun 5, 2018 •
edited
Loading

Currently, the function visualize_attention receives data with the length B and output att_ws with numpy format and the size (B, Lmax, Tmax).
And if use_visualization == True, show the attention weights using matplotlib.
If you have any suggestions, please feedback.

sw005320
sw005320 reviewed Jun 5, 2018
View reviewed changes
Copy link
Copy Markdown
Contributor

@sw005320 sw005320 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Can you add an option to enable it? Also, the use case would be to save the attention weight for every epoch, and see how it behaves. So, it would be better to have such storing functions.

@kan-bayashi
Copy link
Copy Markdown
Member Author

kan-bayashi commented Jun 6, 2018

I implemented attention save function as chainer extension.
I'm not sure how many samples should be saved as figure (currently, only 1).

sw005320
sw005320 reviewed Jun 8, 2018
View reviewed changes
Comment thread src/nets/e2e_asr_attctc_th.py Outdated
CTC_LOSS_THRESHOLD = 10000
CTC_SCORING_RATIO = 1.5
MAX_DECODER_OUTPUT = 5
MAX_SHOW_ATTENTION = 1
Copy link
Copy Markdown
Contributor

@sw005320 sw005320 Jun 8, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how about controlling this as an option? For example, --save-attention can accept the number of attention images (default 0).

@kan-bayashi
updated attention visualization function
c18cbc4 
- remove visulization option from E2E
- modify save-attention option to get integer value
- modify default save directory
@kan-bayashi
modified test script
e11fbcd
@kan-bayashi
fixed flake8 warning
c77bf2b
@sw005320
Copy link
Copy Markdown
Contributor

sw005320 commented Jun 10, 2018

LGTM
You can merge it.

@kan-bayashi kan-bayashi merged commit a0715d3 into espnet:master Jun 11, 2018
@sw005320
Copy link
Copy Markdown
Contributor

sw005320 commented Jun 11, 2018

Very cool!
Thanks!

@sw005320 sw005320 mentioned this pull request Jun 11, 2018
Closed
@kan-bayashi kan-bayashi mentioned this pull request Jun 12, 2018
Closed
@kan-bayashi kan-bayashi deleted the att_vis branch August 28, 2018 09:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@sw005320 sw005320 sw005320 left review comments

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@kan-bayashi @sw005320