Hi author! @Bahuia
Thank you very much for your fix. However, we have some questions regarding the results on WikiSQL and would appreciate your insights.
-
In the revised WikiSQL results, we noticed that all values for the BWT metric are positive, which we also observed in our own replication. This seems somewhat unexpected and inconsistent with the results reported in the paper, unless the model demonstrates a very strong generalization ability.
https://github.com/Bahuia/SSCL-Text2SQL/blob/e55e111e2e916b0a46f08c12cbd0d710049ecc91/wikisql.log#L12543
-
Simultaneously, we noticed that on WikiSQL, SFNET's performance even surpasses the ORACLE method, which appears to be a quite intriguing discovery. However, the reasons behind this phenomenon are not entirely clear to us. Could you please provide further insights on this matter?
https://github.com/Bahuia/SSCL-Text2SQL/blob/e55e111e2e916b0a46f08c12cbd0d710049ecc91/wikisql.log#L12544
Originally posted by @tom68-ll in #3 (comment)
Hi author! @Bahuia
Thank you very much for your fix. However, we have some questions regarding the results on WikiSQL and would appreciate your insights.
In the revised WikiSQL results, we noticed that all values for the BWT metric are positive, which we also observed in our own replication. This seems somewhat unexpected and inconsistent with the results reported in the paper, unless the model demonstrates a very strong generalization ability.
https://github.com/Bahuia/SSCL-Text2SQL/blob/e55e111e2e916b0a46f08c12cbd0d710049ecc91/wikisql.log#L12543
Simultaneously, we noticed that on WikiSQL, SFNET's performance even surpasses the ORACLE method, which appears to be a quite intriguing discovery. However, the reasons behind this phenomenon are not entirely clear to us. Could you please provide further insights on this matter?
https://github.com/Bahuia/SSCL-Text2SQL/blob/e55e111e2e916b0a46f08c12cbd0d710049ecc91/wikisql.log#L12544
Originally posted by @tom68-ll in #3 (comment)