, Modèles : bert-large, uncased et bert-base, multilingual cased 13

, nous cherchons à sélectionner le meilleur type de plongement lexical pour une tâche de détection non-supervisée d'événements dans un flux de tweets, que nous modélisons par un clustering dynamique. Nous montrons, sur un corpus en anglais et un corpus en français, qu'une représentation des tweets par tf-idf permet d'obtenir les meilleurs résultats par rapport à Word2Vec, BERT, ELMo, Universal Sentence Encoder ou Sentence-BERT. Nous montrons également qu'un fine-tuning sur un corpus de quelques centaines de paires de phrases annotées selon leur similarité thématique améliore de deux points les résultats de Sentence-BERT, /UKPLab/sentence-transformers. Modèle : bert-large-nli-stsb-mean-tokens Dans cet article

J. Allan, V. Lavrenko, D. Malin, and R. Swan, Detections, bounds, and timelines: Umass and tdt-3, Proc. of Topic Detection and Tracking workshop, pp.167-174, 2000.

H. Becker, M. Naaman, and L. Gravano, Beyond trending topics: Real-world event identification on twitter, Fifth international AAAI conference on weblogs and social media, 2011.

D. Cer, Y. Yang, S. Kong, N. Hua, N. Limtiaco et al., Universal sentence encoder, 2018.

W. Che, Y. Liu, Y. Wang, B. Zheng, and T. Liu, Towards better UD parsing: Deep contextualized word embeddings, ensemble, and treebank concatenation, Proc. of the CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, pp.55-64, 2018.

A. Conneau, D. Kiela, H. Schwenk, L. Barrault, and A. Bordes, Supervised learning of universal sentence representations from natural language inference data, 2017.
URL : https://hal.archives-ouvertes.fr/hal-01897968

J. Devlin, M. Chang, K. Lee, and K. Toutanova, Bert: Pre-training of deep bidirectional transformers for language understanding, 2018.

F. Fleuret and H. Sahbi, Scale-invariance of support vector machines based on the triangular kernel, 3rd International Workshop on Statistical and Computational Theories of Vision, pp.1-13, 2003.
URL : https://hal.archives-ouvertes.fr/inria-00071984

F. Godin, B. Vandersmissen, W. De-neve, and E. R. Van-de-walle, Multimedia lab @ acl wnut ner shared task: Named entity recognition for twitter microposts using distributed word representations, Proc. of Workshop on Noisy User-generated Text, pp.146-153, 2015.

Z. S. Harris, Distributional structure. Word, vol.10, pp.146-162, 1954.

M. Hasan, M. A. Orgun, and R. Schwitter, TwitterNews: Real time event detection from the Twitter data stream, PeerJ PrePrints, 2016.

J. Johnson, M. Douze, and H. Jégou, Billion-scale similarity search with gpus, IEEE Transactions on Big Data, 2019.

R. Kiros, Y. Zhu, R. R. Salakhutdinov, R. Zemel, R. Urtasun et al., Skip-thought vectors, Advances in neural information processing systems, pp.3294-3302, 2015.

A. J. Mcminn, Y. Moshfeghi, and J. M. Jose, Building a large-scale corpus for evaluating event detection on twitter, Proc. of ACM-CIKM, pp.409-418, 2013.

T. Mikolov, K. Chen, G. Corrado, and E. J. Dean, Efficient estimation of word representations in vector space, 2013.

J. Pennington, R. Socher, and C. Manning, Glove: Global vectors for word representation, Proc. of EMNLP, pp.1532-1543, 2014.

M. E. Peters, M. Neumann, M. Iyyer, M. Gardner, C. Clark et al., Deep contextualized word representations, 2018.

S. Petrovi?, M. Osborne, and V. Lavrenko, Streaming first story detection with application to Twitter, Proc. of NAACL, pp.181-189, 2010.

J. J. Randolph, Free-marginal multirater kappa: An alternative to fleiss' fixed-marginal multirater kappa, 2005.

N. Reimers and I. Gurevych, Sentence-bert: Sentence embeddings using siamese bertnetworks, 2019.

Ø. Repp and H. Ramampiaro, Extracting news events from microblogs, Journal of Statistics and Management Systems, vol.21, issue.4, pp.695-723, 2018.

J. Sankaranarayanan, H. Samet, B. E. Teitler, M. D. Lieberman, and J. Sperling, Twitterstand: news in tweets, Proc. of ACM-GIS, pp.42-51, 2009.

S. Jones and K. , A statistical interpretation of term specificity and its application in retrieval, Journal of Documentation, vol.28, issue.1, pp.11-21, 1972.

A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones et al., Attention is all you need, Advances in neural information processing systems, pp.5998-6008, 2017.

A. Wang, A. Singh, J. Michael, F. Hill, O. Levy et al., Glue: A multitask benchmark and analysis platform for natural language understanding, 2018.

Y. Yang, T. Pierce, and J. G. Carbonell, A study of retrospective and on-line event detection, Proc. of ACM-SIGIR, pp.28-36, 1998.